Recently added questions¶
简体中文 · English
Ordered by the date each question was added, including new questions about older papers. Questions also appear in their topic banks.
4253 questions · 1061 papers
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Method comparison
When the draft uses top-k support, how does renormalizing the teacher there change the comparison?
SpecOPD: Branch-Parallel On-Policy Distillation for Speculative Decoding
Show answer and example
Answer: It discards teacher probability on tokens the draft cannot propose, changing the acceptance-overlap objective. SpecOPD retains full-vocabulary teacher normalization. Outside draft support, q is zero but the missing teacher mass still matters. Replacing the teacher by a top-k conditional distribution is not an innocent computational shortcut.
Illustrative example: Teaching example: q=[.5,.5,0] and p=[.4,.4,.2] have overlap .8. Renormalizing the first two teacher entries invents a perfect match, incorrectly changing loss .2 to zero.
Sections in the Chinese source: 核心公式, 结果、负控与边界. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Core mechanism
Why does reordering identical token-overlap values change the SpecOPD loss?
SpecOPD: Branch-Parallel On-Policy Distillation for Speculative Decoding
Show answer and example
Answer: E2E-TV sums overlap products from the beginning of each branch to every position. Early mismatch reduces every subsequent prefix term; a final mismatch affects only the final term. It is not mean token TV. For history-dependent proposals it is a training surrogate, not an unbiased serving-acceptance estimate.
Illustrative example: Teaching example: Overlaps [0.6,1,1] produce prefix values 0.6,0.6,0.6; [1,1,0.6] gives 1,1,0.6 and a lower loss. These are illustrative, not model measurements.
Sections in the Chinese source: 核心公式, 结果、负控与边界. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Basics
Why is SpecOPD on-policy if its reference remains fixed?
SpecOPD: Branch-Parallel On-Policy Distillation for Speculative Decoding
Show answer and example
Answer: Only the outer reference context is fixed. Each update samples new branches from the current draft after anchors, so teacher-supervised prefixes contain the draft's own history. The complete reference response need not be sampled by the current policy, and collection need not wait for sequential draft-and-verify rounds to finish a response.
Illustrative example: Teaching example: The reference has quiet after a, but the current draft samples short. The next teacher query must condition on a short, not a quiet; these are different histories.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Implementation details
Why check prediction rows separately when the proposal tree already has a causal mask?
SpecOPD: Branch-Parallel On-Policy Distillation for Speculative Decoding
Show answer and example
Answer: Ancestry masking blocks future reference tokens and siblings. But predicting a sampled token from its own hidden-state row already exposes the answer as input. The first token must read its anchor and later tokens their predecessors, with semantic rather than packed positions. Correct masking does not replace correct readout.
Illustrative example: Teaching example: For samples short evening, the anchor predicts short and the short row predicts evening. Scoring evening from its own row still leaks it even if that row sees only its ancestors.
Sections in the Chinese source: 核心公式, 数据协议与代码映射, 结果、负控与边界. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
Does RLiD's variance theorem prove a newly trained LDM must beat a generative LLM?
RLiD: Reinforcement Learning in the Decision Plane for Language Agents and LLMs
Show answer and example
Answer: No. Within one text policy, the identity marginalizes alternative wordings that map to the same action. Given that action, advantage must not depend on wording. It preserves expected updates and removes realization variance, not differences between model parameterizations or training runs, nor temporal-credit noise.
Illustrative example: Teaching example: open drawer and I should open drawer can be marginalized into one action. If reward separately values polite wording, advantage depends on wording and the assumption fails.
Sections in the Chinese source: 核心公式, 数据协议与代码映射. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
Do executable menus and clipped PPO complete RLiD agent training?
RLiD: Reinforcement Learning in the Decision Plane for Language Agents and LLMs
Show answer and example
Answer: No. Old-policy log-probabilities and advantages must come from actual external sampling, stay frozen, and align with the same menu. Local code supplies no LM, compiler, memory, GiGPO, or distillation. With mean-episode-length divided by each episode's length as weights, average weighted decisions without arbitrarily dividing again by total weight.
Illustrative example: Teaching example: A two-step episode with .75 per step and a one-step episode with weight 1.5 both total 1.5. This checks weighting, not the paper's 99.3% ALFWorld full-route result.
Sections in the Chinese source: 数据协议与代码映射, 结果、负控与边界, 高成本完整实验. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
Is KL from uniform to the menu marginal equivalent to making every decision more random?
RLiD: Reinforcement Learning in the Decision Plane for Language Agents and LLMs
Show answer and example
Answer: No. It first averages microbatch action distributions within a menu group, then encourages that label marginal toward uniformity. Individual decisions can remain confident. Per-decision entropy instead encourages each row to spread out, changing the target. External calibration subtraction is another label-bias treatment with a separate update schedule.
Illustrative example: Teaching example: Decisions near [1,0] and [0,1] average to [.5,.5], giving zero marginal KL despite low entropy in each row.
Sections in the Chinese source: 核心公式, 结果、负控与边界. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
How does a RLiD action differ fundamentally from an LLM writing an action command?
RLiD: Reinforcement Learning in the Decision Plane for Language Agents and LLMs
Show answer and example
Answer: RLiD normalizes over compiled executable single-token menu labels and selects an action in one forward pass, without generating or parsing free text. Benefits depend on the menu and observation interface. It does not automatically preserve the extra computation provided by long reasoning, or guarantee stronger performance on every task.
Illustrative example: Teaching example: With only open and take in the menu, label B maps directly to take; free text might contain only an explanation. Legal output follows from an interface contract, not 100% task success.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Core mechanism
Do exactly two percentage points of accuracy gain and exactly 2% lower cost automatically satisfy Strategist's win rule?
Strategist: Evolving Reasoning with Agents
Show answer and example
Answer: No. Improvement requires more than two percentage points or a cost ratio strictly below 0.98; the other metric must remain within its tolerance band. The candidate must win on validation and hidden, then ranks by hidden accuracy gain and cost ratio. The default rule is not a confidence test, even though standard errors are computed.
Illustrative example: Teaching example: d=0.02 and cost ratio 0.98 are ties. d=0 and ratio 0.97 can be a cost win, but a three-percentage-point hidden accuracy loss still prevents selection.
Sections in the Chinese source: 核心公式, 数据协议与实现边界. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Method comparison
Both Strategist and EvalAlloc inspect candidate measurements. Do they optimize the same object?
Strategist: Evolving Reasoning with Agents
Show answer and example
Answer: No. Strategist proposes and composes executable reasoning strategies, retaining structures and experience for later tasks. EvalAlloc mainly determines which SEARCH evaluation cells to execute, how shared references estimate fitness, and how final confirmation works. Cheap observations or high predicted scores do not generate better strategies, and library admission does not establish held-out success.
Illustrative example: Teaching example: evaluating 100 fewer items changes the observation budget. Replacing direct answering with a reusable decomposition-and-review program changes strategy structure. They can be combined, but costs and real held-out success need separate records.
Sections in the Chinese source: 背景与主要改动, 本地结果与负控, 高成本完整实验. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Basics
Does Strategist accumulate model parameters, the current answer, or another cross-task asset?
Strategist: Evolving Reasoning with Agents
Show answer and example
Answer: It freezes executor models and represents reasoning programs as Python blueprints with one execute interface. After evaluation, they enter a cross-task library with measurement cards and training-trace experience. Task prompts are separate from structure; later tasks can compose or reuse structures. This is neither gradient training nor merely editing the current answer.
Illustrative example: Teaching example: a discovered procedure that creates three candidates and reviews them independently can be retested with new prompts in another domain. Saving one problem's final number is not reusable blueprint accumulation.
Sections in the Chinese source: 背景与主要改动, 代码映射. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Implementation details
Why do passing AST, DAG, and paired-pick tests not establish a complete Strategist or Evolve integration?
Strategist: Evolving Reasoning with Agents
Show answer and example
Answer: The module calls no designer or executor model; retrieval scores and candidate evaluations are supplied externally. Explicit prompt elision is only a structural fingerprint, not proof about helper semantics or dynamic dependencies. Paired-pick does not isolate hidden data within a Python process. Actual proposal, execution, retrieval, and information-flow isolation require separate acceptance. The present CPU diagnostic is not promotion-eligible capability.
Illustrative example: Teaching example: synthetic arms with accuracy 0.7 and costs 10 and 8 verify ratio 0.8 and fallback behavior. No model has answered a question, and cross-task search has not been demonstrated.
Sections in the Chinese source: 数据协议与实现边界, 本地结果与负控, 代码映射. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Method comparison
Does exact agreement between ADC and reconstructed distance prove better retrieval than SIDs or dual towers?
WeSearch: A Unified Model-Centric Retrieval Framework for WeChat Search
Show answer and example
Answer: No. It proves computational equivalence for the same quantized representation, not representation quality, cluster coverage or ranking performance. Quality comparisons require trained query models, codebooks and feature FFNs under a fixed candidate budget. The paper's A/B measures the complete production system, not the local lookup kernel.
Illustrative example: Example: If a bad codebook maps every document to one code, ADC can remain exactly equal to explicit distance while documents become indistinguishable.
Sections in the Chinese source: 论文离线与线上效果, 数据协议与结果解释. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Implementation details
Can fresh-document mounting modify the old codebook and call new-array construction an atomic update?
WeSearch: A Unified Model-Centric Retrieval Framework for WeChat Search
Show answer and example
Answer: No. Mounting reuses fixed clusters, rotation and codebooks. Parameter changes can invalidate old codes and require a complete model/index version refresh. Constructing an immutable snapshot preserves the old view but does not implement visibility swaps or distributed atomicity; those remain external service responsibilities.
Illustrative example: Example: Code 1 meant blue before refresh but means red afterward. Old documents would be misinterpreted. Switch the new model with its re-encoded index, rather than replacing only the codebook.
Sections in the Chinese source: 背景与主要改动, 代码映射, 高成本完整实验. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Basics
How does WeSearch cluster selection differ from generating a complete semantic-ID path?
WeSearch: A Unified Model-Centric Retrieval Framework for WeChat Search
Show answer and example
Answer: Clusters constrain the candidate space. After unioning atomic documents from several clusters, document matching remains separate. A cluster need not fully describe a document's semantics. Sequential SID decoding may irreversibly prune a relevant document after an early prefix error. Cluster retrieval can still miss candidates; it does not remove all recall errors.
Illustrative example: Example: An article about travel photography can relate to both topics. Retrieving both clusters and comparing the articles differs from committing to one top-level prefix.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Core mechanism
Is the sum of ADC table lookups already the coarse-ranking score?
WeSearch: A Unified Model-Centric Retrieval Framework for WeChat Search
Show answer and example
Answer: No. Tables contain squared L2 from the rotated query to subspace codewords. Their sum is squared distance, whereas the paper scores with its negative square root. Rankings agree, but numerical distances and thresholds are not interchangeable. Reconstructed vectors also remain in rotated space.
Illustrative example: Example: Squared subspace distances 9 and 16 sum to 25. The score is -5, not -25; using a threshold of -10 would select different candidates.
Sections in the Chinese source: 核心公式, 代码映射. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Core mechanism
Why combine backward input constraints with VREG inspection inside intact fusions?
XLA from the Trenches: Correctness on TPU Workloads at Scale
Show answer and example
Answer: Backward constraints prevent invalid inputs such as negative square roots or division by zero from creating false differences. Macro isolation only finds a fusion; lowering faults inside it still require register intermediates. Splitting it changes memory precision and pipelining and can hide the bug, so a correct split graph does not validate the original fusion.
Illustrative example: Teaching example: r=x−f32(f16(x)) becomes zero if VREG rounding is skipped. Splitting for debugging can spill to 16-bit HBM and restore rounding. Inspect the original fusion rather than replacing it.
Sections in the Chinese source: 核心机制与数值例子, 资源与复现边界. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Implementation details
With OpenXLA public, why mark the full paper not reproduced instead of running an A100 substitute?
XLA from the Trenches: Correctness on TPU Workloads at Scale
Show answer and example
Answer: Public macro isolation and evaluation are only part of the pipeline. Production graphs, the internal agent harness, target ISA/MXU simulation, VREG hardware stepping, and physical-TPU confirmation do not come with the framework. A100 has different register, rounding, and compilation semantics; a CUDA example cannot validate TPU defects. Resource and availability mismatch justifies review DONE/not reproduced rather than fabricated success.
Illustrative example: Teaching example: Correctly executing two casts in CUDA does not show that TPU lowering retained them. Matching outputs do not cover the original TPU-generated code.
Sections in the Chinese source: 已核查与未实现, 资源与复现边界. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Method comparison
Why are 142 resolved bugs, 103 under review, and online A/B gains three different kinds of evidence?
XLA from the Trenches: Correctness on TPU Workloads at Scale
Show answer and example
Answer: Resolved counts describe defects closed through engineering review. Pending findings can still be rejected or revised, so they do not make 245 resolved bugs. User A/B requires actual randomized comparisons and business metrics, not inference from bug counts. The paper qualifies as Google systems infrastructure, not as industrial recommendation evidence passing an A/B gate.
Illustrative example: Teaching example: Of 10 submitted patches, if 6 landed and 4 are pending, 6 are completed, not 10. Even landing all 10 supplies no randomized CTR experiment.
Sections in the Chinese source: 论文工程与线上证据, 背景与主要改动. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Basics
Why don't successful compilation and crash-free execution establish XLA correctness?
XLA from the Trenches: Correctness on TPU Workloads at Scale
Show answer and example
Answer: Silent miscompilations can return finite, correctly shaped tensors while indexing, rounding, buffers, or synchronization are wrong. Completion and briefly normal loss miss failures with no error signal. The paper uses differential references on valid inputs plus hierarchical isolation, not launch success as correctness evidence.
Illustrative example: Teaching example: A narrow-integer iota wrap can make a write at index 5 also modify 261. Execution completes; comparison against correct indexing semantics exposes it.
Sections in the Chinese source: 背景与主要改动, 核心机制与数值例子. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Core mechanism
Why reproject momentum into the current product space rather than retain the old dense direction unchanged?
ProLand: A New Infeasible Manifold Optimization Method for LoRA Fine-Tuning
Show answer and example
Answer: Changing B and A changes the space realizable by their first-order variations. ProLand projects the previous EMA through the current Π before mixing the current projected gradient, keeping momentum realizable. The thin-factor algorithm reconstructs this quantity from gradients of the same loss; it is not ordinary independent Muon momentum for A and B.
Illustrative example: Teaching example: If old B spans the first coordinate and new B turns toward the second, an old-space direction may no longer be realizable unchanged. Compute Π using the new PB and PA before updating the EMA.
Sections in the Chinese source: 核心公式, 检查与负控. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Implementation details
Why not update A first and use new A for B, or multiply ρ by the learning rate again?
ProLand: A New Infeasible Manifold Optimization Method for LoRA Fine-Tuning
Show answer and example
Answer: Algorithms 1/2 compute DA, DB and Gram error from one old factor pair and apply them simultaneously. Sequential updates add coupling terms. Landing's ρ is an independent step size, with no additional γ. Compact polar must preserve zero singular directions, and numerical ill-conditioning near rank changes must remain explicit.
Illustrative example: Teaching example: With γ=.01 and ρ=.05, the Landing coefficient is .05, not .0005; the latter weakens restoration 100-fold. A full UVᵀ on a zero core can also invent updates under zero gradients.
Sections in the Chinese source: 核心公式, 实现与运行, 检查与负控. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Method comparison
How does polar on A and B separately differ from a product-space polar direction realized through factor updates?
ProLand: A New Infeasible Manifold Optimization Method for LoRA Fine-Tuning
Show answer and example
Answer: Polar is nonlinear and the factors have a scaling gauge. Independent spectral updates do not ensure their combined effect matches the selected product-space direction. ProLand constructs DA=B†Y and DB=(I−PB)YA† from one Y, ensuring BDA+DBA=ΠY and BᵀDB=0.
Illustrative example: Teaching example: Multiplying B by 2 and dividing A by 2 preserves BA but changes independent factor-gradient scales and geometry. Calling polar in both paths does not make their ΔW updates equivalent.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Basics
Why does ProLand optimize while B is temporarily nonorthogonal instead of projecting it back every step?
ProLand: A New Infeasible Manifold Optimization Method for LoRA Fine-Tuning
Show answer and example
Answer: It separates the objective direction for BA from Landing restoration of B's orthogonality. BᵀDB=0 makes the objective update perturb the Gram error only at second order, while an independent correction restores feasibility. Per-step retraction changes the trajectory and is not the same algorithm; temporary infeasibility does not imply stability at arbitrary step sizes.
Illustrative example: Teaching example: A one-dimensional basis with BᵀB=1.08 can move horizontally and then subtract ρB×0.08. Direct normalization to 1 follows a different path. This is not a GLUE result.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Core mechanism
Despite Balanced OT in the title, why not fix every visual column marginal by equality?
Show answer and example
Answer: Eq. 1 sets each word-row sum to 1 but only bounds visual column sums by b; Nothing is uncapped. Thus the dual update is v=min(1,b/(Kᵀu)). Requiring every column sum to equal b would assign mass to unnamed regions, changing the feasible set and interpretation.
Illustrative example: Teaching example: If neither of two words names a visible object and a visual column has capacity .8, nearly all mass 2 may legally go to Nothing and almost none to the region. Do not force reference just to spend the .8 capacity.
Sections in the Chinese source: 核心公式, 检查与负控. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Implementation details
Why not renormalize rows after eight Sinkhorn steps, and why detach the plan in alignment loss?
Show answer and example
Answer: Final column scaling enforces upper capacities, but finite-iteration row sums can differ from 1. Row renormalization can exceed capacities again, so expose residuals. Eq. 3 fixes P and backpropagates only through C, giving gradient λP. V1 does not specify another differentiable route for the Nothing hinge; local coverage reports a fixed-plan value, not full anti-collapse training.
Illustrative example: Teaching example: Renormalizing a row summing to .8 multiplies its visual mass by 1.25 and can violate an already saturated column. With λ=.1, cost gradient is .1P, not differentiation through eight scaling iterations.
Sections in the Chinese source: 核心公式, 检查与负控, 成本判断与复现边界. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Method comparison
How does GAVAGAI reduce to row softmax when capacities are inactive, and does that erase the method's distinction?
Show answer and example
Answer: If every visual capacity is at least the word count n, column multipliers remain 1, giving row softmax including Nothing. Removing Nothing then recovers forced visual reference. This degeneration is a useful check, not equivalence under active capacities or evidence of SAYCam improvement.
Illustrative example: Teaching example: With 3 words and capacity 3 per column, no visual column can exceed its cap. Changing that cap to .4 couples row and column scalings as words compete for the region.
Sections in the Chinese source: 核心公式, 检查与负控, 论文离线与线上效果. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Basics
Why does GAVAGAI need both Nothing and region capacities?
Show answer and example
Answer: Nothing lets nonreferential words avoid forced assignment to backgrounds; capacities stop a few persistent regions from absorbing many words. Capacity without Nothing still forces mass onto visual regions. Nothing without capacities cannot limit concentration on one background. They address different failures.
Illustrative example: Teaching example: An utterance contains hungry and ball while the image contains a ball and hand. Hungry can go to Nothing and ball to the ball; a hand capacity also prevents many unrelated words from accumulating there. These assignments are illustrative, not measured.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Basics
Why is finding a similarly titled page after hitting a login wall not necessarily successful AgentHeal recovery?
AgentHeal: Evaluating Goal-Preserving Resource Recovery of Web Agents under Access Barriers
Show answer and example
Answer: It may be another work, an unverifiable look-alike, or an older version. AgentHeal preserves the original objective: prefer an exact-version copy when available; disclose differences when only other versions remain; otherwise abstain and diagnose the barrier, asking the user for help when necessary. Topic similarity is not equivalence.
Illustrative example: Teaching example: a requested metric from paper v2 cannot silently be replaced by v1. Use an institutional v2 copy if one exists; if only v1 remains, explicitly disclose the version difference.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Method comparison
AgentPerturb requires solvability after perturbation, whereas AgentHeal includes unrecoverable tasks. Is that contradictory?
AgentHeal: Evaluating Goal-Preserving Resource Recovery of Web Agents under Access Barriers
Show answer and example
Answer: They target different behaviors. AgentPerturb compares paired robustness under the same objective and must not count destroyed solvability as capability loss. AgentHeal explicitly labels unrecoverable regimes to assess honest abstention, accurate diagnosis, and policy compliance rather than demand nonexistent answers. Recoverable and unrecoverable cases need separate reporting, not one undifferentiated answer-accuracy score.
Illustrative example: Teaching example: deleting the only evidence while still requiring the original number is an unfair perturbation. Labeling that condition unsolvable and testing refusal to fabricate instead measures recovery reliability.
Sections in the Chinese source: 背景与主要改动, 核心公式, 本地结果与负控. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Implementation details
The local diagnostic knows work, version, and reachable resources. Does it reveal the recovery path to an agent?
AgentHeal: Evaluating Goal-Preserving Resource Recovery of Web Agents under Access Barriers
Show answer and example
Answer: Those are independent evaluator annotations. The module calls no actor and must not supply them as actor observations. Source attribution and per-action policy judgments also come from external evaluators rather than local string rules. TS=0.5 across four synthetic truth-table cases verifies the decision contract, not agent capability. Real browser evaluation still needs separation of actor, gold, and judge.
Illustrative example: Teaching example: directly providing copy_id to a model before scoring recovery leaks the path. An independent evaluator should inspect visits and attributed sources only after the model submits.
Sections in the Chinese source: 数据协议与实现边界, 本地结果与负控, 代码映射. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Core mechanism
Why must AgentHeal's exact-copy and alternative-version source conditions remain different?
AgentHeal: Evaluating Goal-Preserving Resource Recovery of Web Agents under Access Barriers
Show answer and example
Answer: Equation 1 requires the attributed source set to intersect available exact copies. When only other versions remain, the nonempty source set must lie entirely within those versions and declare other. Any prohibited action multiplies final success by zero. Strengthening the first branch to copies only, or relaxing the second to one version somewhere, changes the paper.
Illustrative example: Teaching example: with only v1 reachable, mixing v1 and a decoy fails even with an other declaration. When v2 is available, Equation 1 itself only checks that v2 is among the sources; it does not prove answer semantics are correct.
Sections in the Chinese source: 核心公式, 数据协议与实现边界. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Basics
How does MPOPD mixing a rollout policy differ from mixing a student target?
Proposing Better Rollouts for On-Policy Distillation
Show answer and example
Answer: A learned proposal changes which prefixes are visited. The student still minimizes reverse KL to the teacher and corrects the old proposal/student gap. Training against a mixed target changes the loss without necessarily visiting teacher-supported paths.
Illustrative example: Example: Reweighting observed bad paths cannot generate a rare good path that was never collected. A changed proposal may sample it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Implementation details
Why don't real gradients establish full MPOPD integration?
Proposing Better Rollouts for On-Policy Distillation
Show answer and example
Answer: The tensor kernel has no sampler, LM copy or teacher call. Each round must reset proposal from the updated student and optimize it toward the new geometric target. Student KL and target are renormalized on the same teacher top-k support. Numeric checks do not verify trajectory origins, resets or extended training.
Illustrative example: Example: Cached old prefixes backpropagate but aren't current proposal rollouts. A small CUDA tensor check is not accuracy on 16 reasoning benchmarks.
Sections in the Chinese source: 数据协议与实现边界, 代码映射, 高成本完整实验. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Core mechanism
Which importance weight multiplies the student's KL at position j?
Proposing Better Rollouts for On-Policy Distillation
Show answer and example
Answer: Eq. 7 uses the one-step ratio w[j−1], with weight 1 at the first prefix. Use frozen old student and proposal probabilities of the sampled token and clip. Neither current-token ratio nor cumulative product is correct; no recomputation by the updating student.
Illustrative example: Example: Ratios 4, 0.25, 4 yield KL weights [1,4,0.25], not [4,0.25,4] or [1,4,1].
Sections in the Chinese source: 核心公式, 本地结果与负控. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Method comparison
Does fixed-eta geometric mixing equal a probability average or a fixed KL radius?
Proposing Better Rollouts for On-Policy Distillation
Show answer and example
Answer: Neither. It adds weighted log probabilities before normalization, not probabilities. Distributions vary by prefix, so the same eta produces different KL movement. A fixed local radius can imply prefix-specific eta; the global setting is a computational compromise.
Illustrative example: Example: Eta 0.4 barely moves a prefix where teacher and student agree but moves more mass when they conflict.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Implementation details
Can the first group in a batch decide the CONCUR-A ceiling from only previous history?
CONCUR: Concordant Auxiliary Rewards for Multi-Task, Multi-Reward Reinforcement Learning
Show answer and example
Answer: No. Algorithm 1 merges every valid reading in the batch before sharing one policy per task/auxiliary and projecting groups. Sequential updates would depend on group order. Final whitening spans the full multi-task batch, not each group. A tensor check is not LLM lift.
Illustrative example: Example: History has 60 valid positives and group two supplies the 61st; group one also uses the merged current trust state.
Sections in the Chinese source: 核心公式, 数据协议与实现边界, 代码映射. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Basics
Why can standardized rewards violate primary priority despite gating raw auxiliary rewards?
CONCUR: Concordant Auxiliary Rewards for Multi-Task, Multi-Reward Reinforcement Learning
Show answer and example
Answer: The raw gate constrains auxiliary scores for failures. Independent centering/scaling can amplify small auxiliary gaps enough to reverse primary order. At a primary ceiling the primary advantage is zero, so auxiliary can determine the update. CONCUR constrains standardized terms, not just the raw gate.
Illustrative example: Example: Two correct answers differ slightly in length reward. Standardization can give length unit variance while primary advantage is zero; this is not evidence of learning more correct answers.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Core mechanism
Why sort by primary then auxiliary before PAVA without averaging all equal primary scores?
CONCUR: Concordant Auxiliary Rewards for Multi-Task, Multi-Reward Reinforcement Learning
Show answer and example
Answer: Monotonicity only constrains strictly higher primary levels, not equal non-ceiling primary values. Sorting ties by auxiliary lets PAVA fix cross-primary violations only. Equal outputs are required at the ceiling only when that tie is enabled.
Illustrative example: Example: Primary [0.3,0.3,0.7], auxiliary [-1,0,1] is already feasible. Replacing the two first values with −0.5 adds a constraint.
Sections in the Chinese source: 核心公式, 本地结果与负控. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Method comparison
Does CONCUR-A exceeding two standard errors prove auxiliary reliability at the ceiling?
CONCUR: Concordant Auxiliary Rewards for Multi-Task, Multi-Reward Reinforcement Learning
Show answer and example
Answer: No. It measures concordance of passing pairs with different primary values, releasing ties after at least 61 valid readings and mean greater than two standard errors. The evolving distribution is not IID, and ceiling pairs are not directly measured. This is not a significance level or universal guarantee.
Illustrative example: Example: Concordance can reverse later and drop below the threshold; rebind the ceiling rather than treating the first release as permanent.
Sections in the Chinese source: 核心公式, 数据协议与实现边界. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Core mechanism
Why not gate the current state, and why require true mismatched discrimination?
SWIM: Learning What Matters from Behaviors for JEPA World Models
Show answer and example
Answer: The current state is full action context; the paper gates only future displacement. Fitting positives alone can predict actions from current context and ignore displacement. A cyclic mismatch changes the future while holding current state/action fixed, encouraging behavior-relevant displacement.
Illustrative example: Example: Keep the current position and move-right action. For a negative, take the next batch row's future minus this position. Replacing the current state gives a spurious context cue.
Sections in the Chinese source: 核心公式, 检查与负控. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Implementation details
How do tests distinguish L1 shrinkage from learning behavior, and prevent future state leakage?
SWIM: Learning What Matters from Behaviors for JEPA World Models
Show answer and example
Answer: Check gate gradients from the behavioral term alone, then real optimizer updates; parameter change alone is insufficient. In stage two state distance is stop-gradient. Future state constructs offline displacements only and must not be available to deployed planning. Constant-distance Pearson is undefined and should be rejected, not declared valid via arbitrary epsilon.
Illustrative example: Example: λsum(g) shrinks gates even with actions ignored, so also test λ=0. If all frame distances are zero, reject the zero-denominator correlation rather than report success 1.
Sections in the Chinese source: 本地复现, 检查与负控. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Method comparison
Are multiplying state by g or sqrt(g), or standardizing again afterward, equivalent?
SWIM: Learning What Matters from Behaviors for JEPA World Models
Show answer and example
Answer: No. Square root scaling yields coefficient g in squared distance; scaling by g gives g². Gates weight already-standardized coordinates; re-standardization cancels scaling. Overall gate scale cancels from Pearson correlation and can be absorbed by the inverse model, so absolute gate magnitudes are not uniquely identifiable.
Illustrative example: Example: Difference 2 with g=0.25 should give 0.25×4=1. Multiplying by sqrt(g)=0.5 before squaring gives 1; multiplying by g gives 0.25. Rescaling the standard deviation afterward cancels the 0.5.
Sections in the Chinese source: 核心公式, 本地复现. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Basics
Does SWIM learn a robot policy or which state coordinates matter for distance?
SWIM: Learning What Matters from Behaviors for JEPA World Models
Show answer and example
Answer: It learns continuous coordinate weights from recorded actions and uses them for world-model geometry. Inverse action prediction only trains the weights. Deployment still needs image encoding, latent dynamics and planning such as CEM. Local coverage is the gate objective and metric, not a robot policy.
Illustrative example: Example: Block position can receive more weight than background light color. That changes distance, not the robot's grasp trajectory.
Sections in the Chinese source: 背景与主要改动, 本地复现. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Core mechanism
Why does Eq. 7 distinguish the second moment from covariance?
Multi-State Linear Attention with Generalized Second-Order Routing
Show answer and example
Answer: It stores mean(kkᵀ), without centering, then computes variance qᵀΣq−(qᵀμ)². Passing covariance and subtracting again double-centers; treating the raw moment as variance includes mean strength. The local reference directly measures dot-product variance.
Illustrative example: Example: Keys [1,3] have mean 2, moment 5 and variance 1. Using 5 overcounts by 4; subtracting 4 from centered 1 yields the invalid −3.
Sections in the Chinese source: 核心公式, 检查与负控. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Implementation details
Why only expose completed segments to local queries?
Multi-State Linear Attention with Generalized Second-Order Routing
Show answer and example
Answer: Full-segment means and moments include future tokens for queries inside that segment. A full model could maintain prefix moments and recurrent state; the local path does not. It exposes completed segments only and rejects calls with none visible, rather than leaking future statistics.
Illustrative example: Example: Segment start 4 length 4 covers positions 4–7. It is not visible at 6, only at 7; reading all keys at 4 leaks positions 5–7.
Sections in the Chinese source: 本地复现, 检查与负控. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Method comparison
Why not mix logits then apply one softmax instead of mixing two softmax distributions?
Multi-State Linear Attention with Generalized Second-Order Routing
Show answer and example
Answer: Softmax is nonlinear. MERLIN normalizes mean and variance routes separately and mixes probabilities with gamma. Mixing logits first changes competition and normalization even though both outputs sum to one.
Illustrative example: Example: Mean logits [4,0], variance logits [0,0], gamma 0.5 yield [0.741,0.259]; softmax of mixed logits [2,0] gives [0.881,0.119].
Sections in the Chinese source: 核心公式, 检查与负控. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Basics
Why can two key segments with the same mean require different attention routing?
Multi-State Linear Attention with Generalized Second-Order Routing
Show answer and example
Answer: Mean alignment can hide strong key matches canceled by others. MERLIN also retains raw second moments and recovers query-direction alignment variance, mixing it with mean routing. It enriches segment routing without reconstructing full per-token attention.
Illustrative example: Example: Keys [-2,2] and [0,0] both average zero. Query 1 gives alignment variances 4 and 0, invisible to mean routing.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Core mechanism
Why rescale a ridge direction to μᵀu=1 rather than use unconstrained least squares?
Folding Attention Biases into Keys Makes Compacted KV Caches Run on Standard Kernels
Show answer and example
Answer: Unconstrained LS trades mean offset against projection spread and may shrink all biases. The paper rescales a ridge-stabilized inverse covariance direction to force mean projection one and minimize spread under that constraint. Covariance divides by N and ridge uses its trace; replacing N with N−1 changes the objective.
Illustrative example: Example: With one-dimensional μ=2, variance 1 and no ridge, unconstrained LS gives 2/5 and mean projection 0.8. The constrained direction 1/2 gives exactly 1.
Sections in the Chinese source: 核心公式, 检查与负控. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Implementation details
Why actually cast keys to BF16 before refitting values?
Folding Attention Biases into Keys Makes Compacted KV Caches Run on Standard Kernels
Show answer and example
Answer: Least squares must see attention from the rounded keys. Fitting with high-precision keys then rounding changes the matrix and invalidates the intended normal equations. The local path casts before building A and returns fit-precision coefficients; cache installation, value rounding and backend correctness need separate validation.
Illustrative example: Example: BF16 maps 1.003 to 1.0. Fitting attention with 1.003 compensates an attention table that deployment will not see.
Sections in the Chinese source: 核心公式, 本地复现, 检查与负控. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Method comparison
Why is FoldBias more informed than deleting bias, but not better for every query?
Folding Attention Biases into Keys Makes Compacted KV Caches Run on Standard Kernels
Show answer and example
Answer: Deletion multiplies bias by zero; folding multiplies by q·u and can be closer to one if future queries resemble calibration. Drift can be worse. The bound also needs bias range and value diameter and compares unrounded keys at fixed values, not a refitted serving stack.
Illustrative example: Example: q·u=0.9 leaves 10% bias error; q·u=−2 gives coefficient 3 vs deletion's 1. This is not LLM accuracy.
Sections in the Chinese source: 背景与主要改动, 核心公式, 论文离线与线上效果. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Basics
Why can't a static bias folded into a key be exact for every future query?
Folding Attention Biases into Keys Makes Compacted KV Caches Run on Standard Kernels
Show answer and example
Answer: A key shift contributes a linear function of query, whereas nonzero bias is query-independent. FoldBias fits a direction u so q·u is near 1 on context queries; future drift can increase error. It is distribution-dependent, not an identity.
Illustrative example: Example: A zero query produces zero logit change from any key shift, not bias 0.8. A query satisfying q·u=1 can express that bias exactly.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Method comparison
Why aren't equally sized independent random item sets equivalent to a shared core?
Where Should the Evaluation Budget Go in LLM-Driven Agent Scaffold Evolution?
Show answer and example
Answer: Common items supply paired covariance, helping when candidates agree on most items and set difficulty varies. Frozen item model and prior put successive generations on one scale. Refitting per candidate breaks comparability. This benefit depends on candidate correlation, not guaranteed variance reduction.
Illustrative example: Example: A faces hard items and B easy ones. A common informative core is more comparable than merely requiring 60 items each.
Sections in the Chinese source: 核心公式, 数据协议与实现边界. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Basics
If cheap search feedback is sufficient, why spend extra on final confirmation?
Where Should the Evaluation Budget Go in LLM-Driven Agent Scaffold Evolution?
Show answer and example
Answer: Search needs comparable low-cost guidance; final selection must separate near-tied candidates. EvalAlloc uses sparse shared-core search and paired confirmation that reuses cache, with real observations for the final prefix rather than perpetual imputation.
Illustrative example: Example: A predicted difference of 0.01 can guide exploration but should not decide the champion by itself.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Core mechanism
Can partial credit 0.9 add 0.9 to an item's acquisition success count?
Where Should the Evaluation Budget Go in LLM-Driven Agent Scaffold Evolution?
Show answer and example
Answer: No. Acquisition counts only full credit a_i and uses a Beta posterior for conservative p(1−p) informativeness. Partial grades are not full successes there but remain scores for the response model. Mixing the stages changes the shared core.
Illustrative example: Example: Ten 0.9 scores yield a=0, n=10, not a=9; 0.9 still enters model fitting and final scores.
Sections in the Chinese source: 核心公式, 本地结果与负控. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Implementation details
Why retain the search-cache union and check candidate IDs in cross-fitting?
Where Should the Evaluation Budget Go in LLM-Driven Agent Scaffold Evolution?
Show answer and example
Answer: Already executed cells need not be paid for again; fill missing common-prefix items and retain candidate-specific observations. Separate two-fold refit predictions prevent candidates from training their own correction. The correction is zero when all items are observed.
Illustrative example: Example: Keep A's cached item 4 even if prefix is 0/1/2. If a round needs four new cells and only three remain, reject the round; never read hidden complete scores to select a winner.
Sections in the Chinese source: 核心公式, 数据协议与实现边界, 本地结果与负控. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
Why treat context overflow differently from missing capability?
EMMA: Structured Episodic Memory for Multi-Agent Workflow Synthesis
Show answer and example
Answer: Overflow can mean a step is too broad. Split incomplete steps and resume without discarding finished work. A new specialist needs evidence of missing capability or repeated overflow; repeated overflow calls for changing the decomposition, not simply changing prompts/skills.
Illustrative example: Example: If steps one and two already produced a table and step three is too large, keep the table and subdivide step three among existing agents.
Sections in the Chinese source: 背景与主要改动, 核心公式, 代码映射. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why require one edited entry and retain at least 60% of old prompt lines?
EMMA: Structured Episodic Memory for Multi-Agent Workflow Synthesis
Show answer and example
Answer: Single-entry and ownership scope bound location; retention, length and pool capacity bound magnitude. Only agents involved in the failed attempt may change. New entries must actually be used in the next plan. Structural validity is not better task performance.
Illustrative example: Example: Keeping two of five lines and replacing the role is rejected. Keeping three and adding a local check reaches 60%, but still needs an external rerun.
Sections in the Chinese source: 核心公式, 本地结果与负控. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Why doesn't EMMA turn every failure into a new skill?
EMMA: Structured Episodic Memory for Multi-Agent Workflow Synthesis
Show answer and example
Answer: Ownership depends on the error. Use task-local plan guidance for one execution slip, agent prompts for general habits, skills for missing procedures and controller changes for routing. Experience at the wrong scope can fail to trigger or interfere elsewhere.
Illustrative example: Example: Checking units can be reusable. One misclick does not require a permanent unit-check skill or a new agent.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
Do passing local guards prove no memory leakage and a stronger agent?
EMMA: Structured Episodic Memory for Multi-Agent Workflow Synthesis
Show answer and example
Answer: No. The code checks ownership, capacity, retention, rotation, lifecycle and concatenation. Independent external services judge semantic deduplication, real specialization and leakage; the component cannot prove those judgments correct. No evaluated actor, LLM reflection or real task traces.
Illustrative example: Example: A service may incorrectly mark an answer-bearing proposal safe. A Boolean interface cannot detect that; local rejection counts are not AppWorld success rates.
Sections in the Chinese source: 数据协议与实现边界, 本地结果与负控, 高成本完整实验. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why isn't prefix diversity just subtracting a constant for duplicate next tokens?
PinGER: Token-Efficient Generative Retrieval with Semantic IDs
Show answer and example
Answer: Penalty depends on longest common prefix depth between parent beams and accumulated chosen slots, including the same parent. Deeper shared prefixes penalize repeated subtrees more. The next token alone drops hierarchy.
Illustrative example: Example: Parents (1,2) and (1,3) share depth one; (1,2) with itself shares depth two. After picking its child, penalties differ.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Why does PinGER DPO accumulate every prefix and retain an NTP anchor?
PinGER: Token-Efficient Generative Retrieval with Semantic IDs
Show answer and example
Answer: Per-level loss gives earlier SID decisions preference signal; using only full IDs changes the objective. NTP limits drift. Reference margin offsets the logistic input, not a hard stop-gradient for correct ranking. Local checks are formula-only.
Illustrative example: Example: For three levels sum losses for prefixes 1, 1–2 and 1–3, then add NTP. Tripling only the final loss differs.
Sections in the Chinese source: 核心公式, 本地复现与边界. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
Does improved online interest coverage guarantee higher offline recall?
PinGER: Token-Efficient Generative Retrieval with Semantic IDs
Show answer and example
Answer: No. Increasing lambda from 0 to 3 increases distinct@1 but lowers R@100 in the paper. A stronger online penalty can still suit its objectives/settings. Do not conflate offline tables with production A/B under a different beam width.
Illustrative example: Example: 100 items from one category may hit an established preference but miss other interests. Diversity trades metrics off.
Sections in the Chinese source: 论文离线与线上效果. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Why keep SIDs during PinGER training but remove historical SIDs at serving?
PinGER: Token-Efficient Generative Retrieval with Semantic IDs
Show answer and example
Answer: Asymmetric masking sends historical item information forward only through dense tokens. Historical SID positions are not read by other items or dense states, so removing those positions does not alter retained dense states. Dense input can still incorporate SID features.
Illustrative example: Example: A historical item occupies three SID positions and one dense position during training. Keeping only the dense position can preserve its attention output if inputs match.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why subtract the CPC head if CPC improvement is negative lift?
Reinforcement Learning for Controller Parameters in the Auto-Bidding System at Pinterest
Show answer and example
Answer: Lower CPC is better and below-baseline change remains negative. In Qbu−beta×Qcpc, subtracting negative CPC raises the score. A multi-step CPC backup also uses min rather than max.
Illustrative example: Example: For equal budget score 0, CPC heads −0.1 and 0.2 with beta=1 give fused scores 0.1 and −0.2.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Implementation details
Does implementing top-K softmax reproduce production action probabilities?
Reinforcement Learning for Controller Parameters in the Auto-Bidding System at Pinterest
Show answer and example
Answer: No. Production also needs trained Q, beta scale and post-fusion preprocessing. The component externalizes beta and preprocessing; identity is just a reference. Small tensor tests cannot supply logged propensities or actual outcomes.
Illustrative example: Example: Doubling scores often preserves top-K but sharpens softmax, changing IPS propensities despite the same argmax.
Sections in the Chinese source: 核心公式, 本地复现与边界. The example illustrates the mechanism.
Method comparison
Why can deployed RLBP behave like a contextual bandit despite the DQN name?
Reinforcement Learning for Controller Parameters in the Auto-Bidding System at Pinterest
Show answer and example
Answer: Deployed gamma=0 regresses same-day outcomes with no future Q bootstrap. The proposed multi-step extension independently maxes/mins heads before fusion and need not equal optimal Q of the combined reward. It is not the deployed long-run optimum.
Illustrative example: Example: Reward 1, future Q 100: gamma 0 yields 1, whereas gamma 0.9 changes the objective to 91.
Sections in the Chinese source: 背景与主要改动, 核心公式, 论文离线与线上效果. The example illustrates the mechanism.
Basics
Does RLBP use a neural network to bid in every auction?
Reinforcement Learning for Controller Parameters in the Auto-Bidding System at Pinterest
Show answer and example
Answer: No. It chooses daily PID multipliers and writes a fixed daily lookup table; the original PID controls high-frequency feedback. Production varies P and I over three factors each with D=1, giving nine actions.
Illustrative example: Example: Set P to 1.2 times its default in the morning. Individual bids still respond to PID errors.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Core mechanism
Why threshold the length critic, and how are the two textual gradients combined?
CROP: Token-Efficient Reasoning in Large Language Models via Regularized Prompt Optimization
Show answer and example
Answer: Section 3.3 runs length criticism only when generation strictly exceeds the threshold. Eq. 3 concatenates task critique, newline and length critique per sample for the meta-optimizer, rather than averaging numeric losses. Gradually remove narrative redundancy while keeping required calculations.
Illustrative example: Example: Threshold 40 does not trigger at 40 tokens but does at 41. “Check unit conversion” and “remove the repeated intro” cannot be replaced by scalar 0.7.
Sections in the Chinese source: 核心公式, 数据协议与实现边界. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Implementation details
How does local selection avoid test data or a final unevaluated prompt, and what do callbacks establish?
CROP: Token-Efficient Reasoning in Large Language Models via Regularized Prompt Optimization
Show answer and example
Answer: Selection requires fixed validation revision/IDs matching the frozen CoT length baseline, disjoint training IDs, and no train/test scoring for selection. Each round validates the current prompt before proposing the next, so the final unevaluated one cannot win. Deterministic model/critic fixtures verify gating, concatenation and selection, not real LLM improvement.
Illustrative example: Example: Measured A=0.8 and B=0.9; last proposal C has no validation. Return B, not newest C.
Sections in the Chinese source: 数据协议与实现边界, 本地结果与负控, 高成本完整实验. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Method comparison
What does CROP add beyond accuracy-only prompt tuning, and are token savings lossless?
CROP: Token-Efficient Reasoning in Large Language Models via Regularized Prompt Optimization
Show answer and example
Answer: It adds length feedback and selection by output-token cost. Table 3 includes accuracy drops: Gemini2Flash on ObjectCounting goes 96.0%→94.8% while mean tokens go 101.2→19.6. Report both sides of the tradeoff, not universal losslessness.
Illustrative example: Example: At equal 0.9 accuracy a 50-token prompt can beat a 100-token one. If the short prompt is less accurate, do not report only length.
Sections in the Chinese source: 背景与主要改动, 论文离线与线上效果, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Basics
Does CROP optimize model parameters, individual answers or the system prompt?
CROP: Token-Efficient Reasoning in Large Language Models via Regularized Prompt Optimization
Show answer and example
Answer: It freezes model weights and iterates the system prompt offline. The model generates; task and length critics identify accuracy and redundancy problems; a meta-optimizer rewrites the prompt. Deployment uses the selected prompt for one generation. This is neither hard truncation nor RL weight updates.
Illustrative example: Example: Iteratively changing “explain every step” to “keep the combined calculation and remove repetition” is prompt optimization; cutting any reply after 50 tokens is not.
Sections in the Chinese source: 背景与主要改动, 原论文关键图. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why does Sample–Beam separate sampled reasoning from item beam generation?
Show answer and example
Answer: Serving returns an item set; sampling one response does not match that structure. Sample reasoning, then beam-expand the SID to train RL on a serving-like candidate set. Intent rewards and trajectories are still needed; beam search alone is not the full method.
Illustrative example: Example: Expand ten product codes under one rationale instead of sampling ten unrelated rationale–ID pairs.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Can a 0.58 percentage-point CTR gain be reported as 0.58% relative?
Show answer and example
Answer: No. Percentage points are absolute; relative change depends on baseline CTR. The paper's result is from full deployment. This repository records the work only, without model, private-data training or A/B; don't attribute production lift to a local stub.
Illustrative example: Example: 10%→10.58% is +0.58 points but +5.8% relative. This illustrates units, not the paper baseline.
Sections in the Chinese source: 论文离线与线上效果, 本地复现与边界. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
Does more reasoning always improve PailitaoMMSearch2.0 retrieval?
Show answer and example
Answer: No. The 0.8B No-think variant has highest macro ER; 2B Low-think better balances overall quality and latency, and 4B varies by intent. Consider scale, intent and serving cost rather than generalizing a single optimum.
Illustrative example: Example: A clearly visible exact-match attribute may need no rationale, while a match with several constraints can benefit.
Sections in the Chinese source: 论文离线与线上效果. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Does PailitaoMMSearch2.0 rewrite queries or directly generate items?
Show answer and example
Answer: It consumes the original image and free text and generates three-level SIDs that map to products. Decoding produces candidates. Some deployments still mix them with incumbent retrieval results; end-to-end generation does not remove every production module.
Illustrative example: Example: Upload a shirt and request matching pants; outputs are product IDs, not just a rewritten query for another engine.
Sections in the Chinese source: 背景与主要改动, 核心公式, 论文离线与线上效果. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Core mechanism
Does KL-MOPD compute alpha from the current student or the frozen initial student?
KL-MOPD: Discrepancy-Aware Multi-Objective On-Policy Distillation for Generative Recommendation
Show answer and example
Answer: Calibrate once before training using the frozen initial student's own sampled prefixes. Training loss uses current student prefixes but does not recalculate alpha from current gaps. Dynamic weights would define a different objective.
Illustrative example: Example: Initial gaps 1 and 3 produce fixed weights 1/4 and 3/4. Later gaps of 2 and 1 do not swap them.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Implementation details
Why doesn't a gradient-tested tensor loss establish on-policy recommender training?
KL-MOPD: Discrepancy-Aware Multi-Objective On-Policy Distillation for Generative Recommendation
Show answer and example
Answer: Logits alone don't tell the kernel who sampled a prefix. Initial calibration, current student generation, frozen teacher scoring, SID decoding and real splits require external integration. Only the loss component and small CPU/CUDA mechanism evidence are present.
Illustrative example: Example: The kernel accepts cached old-policy prefixes, but that does not make them samples from the current student.
Sections in the Chinese source: 本地复现与边界. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Method comparison
Is weighting two reverse KL losses equivalent to averaging teacher probabilities and computing one KL?
KL-MOPD: Discrepancy-Aware Multi-Objective On-Policy Distillation for Generative Recommendation
Show answer and example
Answer: Not in general. Weight the separate student-to-teacher KL losses, not an arithmetic mixture of teacher probabilities, and keep KL(student||teacher), not forward KL. Negative controls distinguish the computations without establishing recommendation gains.
Illustrative example: Example: Teachers favor different tokens. An arithmetic mixture keeps mass on both modes; weighted reverse KL involves log teacher probabilities and yields different gradients.
Sections in the Chinese source: 核心公式, 本地复现与边界. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Basics
Why estimate each teacher's difficulty at each SID position in KL-MOPD?
KL-MOPD: Discrepancy-Aware Multi-Objective On-Policy Distillation for Generative Recommendation
Show answer and example
Answer: An item ID needs several correct tokens. The click teacher may be easy to imitate early and the value teacher later. Fixed half weights ignore this; KL-MOPD upweights the harder teacher based on each initial student-to-teacher KL gap.
Illustrative example: Example: A three-part code starts with category and ends with attributes. If a teacher diverges only on category, one sequence-wide weight obscures that difference.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Core mechanism
Why average the lowest-CE positions in a chunk rather than the highest?
Select-TTT: Query-Conditioned Selective Fast Weight Updates for In-Place Test-Time Training
Show answer and example
Answer: CE is mean negative log probability of question tokens under the current hidden state's vocabulary distribution. Lower values mean the position predicts question words better. Averaging the lowest k preserves local relevance more robustly than a single minimum. Reversing it reinforces irrelevant positions.
Illustrative example: Example: For CE 1 and 6, select 1 as more relevant. Repeated question tokens count as many times as they appear.
Sections in the Chinese source: 核心公式, 检查与负控. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Implementation details
Why emit the current chunk before writing its keys and values?
Select-TTT: Query-Conditioned Selective Fast Weight Updates for In-Place Test-Time Training
Show answer and example
Answer: Its outputs read the pre-update state; only later chunks use the new information. Writing first leaks the current update into an earlier output and changes causal order. The local reference tests tensor-state transitions, not real NTP-aligned features, backbone integration or long-dialogue evaluation.
Illustrative example: Example: Start with W=0. Chunk one outputs 0 then writes W=1. Chunk two can read 1; an output of 1 in chunk one would use its write prematurely.
Sections in the Chinese source: 核心公式, 本地复现, 检查与负控. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Method comparison
Why isn't Select-TTT a strictly conserved write-budget allocation?
Select-TTT: Query-Conditioned Selective Fast Weight Updates for In-Place Test-Time Training
Show answer and example
Answer: Softmax multiplied by chunk count sums to that count before clipping. An upper cap can lower the sum. Approximate mean learning-rate preservation is not strict conservation. Renormalization changes Eq. 10 and can violate the per-chunk cap.
Illustrative example: Example: Three chunks can start near [3,0,0]. A cap of 1.2 gives total about 1.2, not 3; this is a formula example, not a QA result.
Sections in the Chinese source: 核心公式, 检查与负控. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Basics
Does Select-TTT change retrieval or the strength of writes to fast weights?
Select-TTT: Query-Conditioned Selective Fast Weight Updates for In-Place Test-Time Training
Show answer and example
Answer: It keeps In-Place TTT chunk writes and scales each chunk's learning rate with question relevance. Scores come from question-token negative log probabilities under the frozen LM head. It neither trains a retriever nor discards all low-scoring chunks. Different questions can induce different multipliers.
Illustrative example: Example: A long conversation may favor different passages for a moving date and food preferences, without labels telling the model where the answer is.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Core mechanism
Why route action and ROI through the residual predictor rather than directly to the structured scoring head?
GUI-JEPA: Candidate-Conditioned JEPA Latents as a Fast Action Policy for Mobile GUI Agents
Show answer and example
Answer: The paper forces candidate-conditioned visual information through the JEPA predictor rather than a shortcut. Action and ROI condition a query; a three-layer Transformer creates a residual latent, followed by ROI and global pooling. Training with a decision loss does not prove accurate future-screen prediction.
Illustrative example: Example: Two click candidates with different ROI content and positions should give different latents. A bypass from raw ROI to the scorer cannot demonstrate predictor utility.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Implementation details
Does a verified three-layer predictor and gradient path mean the repository can operate Android?
GUI-JEPA: Candidate-Conditioned JEPA Latents as a Fast Action Policy for Mobile GUI Agents
Show answer and example
Answer: No. It receives external screen/condition latents and verifies the residual head, candidate masking and weighted listwise loss. It has no encoder, a11y enumeration, MiniLM ranker or executor. Positive labels are loss-only, not forward inputs. A GPU tensor check is not AndroidControl success.
Illustrative example: Example: Backpropagating on two valid candidate latents only checks gradients. Launching an app also requires screen encoding, candidate discovery, execution and observing the next screen.
Sections in the Chinese source: 本地复现, 检查与负控, 高成本完整实验. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Method comparison
How does GUI-JEPA scoring differ from predicting a next state and choosing the one closest to a goal?
GUI-JEPA: Candidate-Conditioned JEPA Latents as a Fast Action Policy for Mobile GUI Agents
Show answer and example
Answer: Latents are decision features trained by listwise action loss; distance to a goal latent is not the value. A randomly initialized predictor can match a dynamics-pretrained one in the paper, so useful decision features are not necessarily good dynamics predictions.
Illustrative example: Example: Confirming may finish a task with little visual change, while an animation looks closer to a target image but is wrong. Action score is not image or latent similarity.
Sections in the Chinese source: 背景与主要改动, 核心公式, 论文离线与线上效果. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Basics
Why doesn't GUI-JEPA generate actions token by token?
GUI-JEPA: Candidate-Conditioned JEPA Latents as a Fast Action Policy for Mobile GUI Agents
Show answer and example
Answer: It enumerates actions from the accessibility tree and conditions visual latents on each candidate, then scores the batch. The action space comes from observations rather than free-form text generation. Speed depends on candidate count and a11y input, which pixel-only agents may lack, so not all GUI systems are directly comparable.
Illustrative example: Example: Score three valid button actions and choose the best, instead of generating JSON and parsing coordinates.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Method comparison
Does better rubric compliance automatically promote the next harness?
When the Critic Falls Behind: Co-Evolving Rubrics and Harnesses for Continual Agent Self-Improvement
Show answer and example
Answer: No. Rubrics diagnose weaknesses and guide proposals, not task outcomes. Candidates must pass external verification and compete with the current harness on independent validation. Only the top-scoring candidate whose improvement strictly clears the threshold replaces it. Final test is excluded.
Illustrative example: Example: A candidate adds many evidence-check lines but does not improve validation success. Keep the current harness rather than promote prettier self-check logs.
Sections in the Chinese source: 核心公式, 数据协议与实现边界. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Implementation details
Does outcome-blind judging conflict with later use of success/failure labels?
When the Critic Falls Behind: Co-Evolving Rubrics and Harnesses for Continual Agent Self-Improvement
Show answer and example
Answer: No. The judge sees behavior only when assigning rubric labels, avoiding hindsight. A separate statistical layer then joins those labels with outcomes to form the six-cell table. The component accepts both inputs but cannot prove an external judge was blinded or claim full evaluation integration.
Illustrative example: Example: Seal the label for whether the target file was checked before revealing test success; don't reinterpret an ambiguous check after learning the run failed.
Sections in the Chinese source: 背景与主要改动, 数据协议与实现边界, 本地结果与负控. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Basics
If an agent can already edit its harness, why update the rubric too?
When the Critic Falls Behind: Co-Evolving Rubrics and Harnesses for Continual Agent Self-Improvement
Show answer and example
Answer: A repaired harness changes its failure distribution. Fixed criteria may track resolved faults and miss new ones. RuHarness reevaluates criteria against current trajectories while keeping the task objective fixed, then uses criteria that still diagnose weaknesses to guide edits.
Illustrative example: Example: Initially it forgot search; later it searches but submits with insufficient evidence. Rewarding any search call misses the new failure.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Core mechanism
Why compute discrimination, coverage and predictiveness from one six-cell posterior?
When the Critic Falls Behind: Co-Evolving Rubrics and Harnesses for Continual Agent Self-Improvement
Show answer and example
Answer: They are correlated because they come from the same success/failure × Violated/Satisfied/N/A table. Add 0.5 to each cell, draw 2,000 joint Dirichlet samples, recompute all three for each draw and take the 0.2 utility quantile. Independent draws would discard correlations. Trajectory-set N and Red are fixed during sampling.
Illustrative example: Example: Two observed violations both fail, giving a high point estimate. A lower posterior quantile reflects that two hits are not hundreds of stable observations.
Sections in the Chinese source: 核心公式, 数据协议与实现边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
Why can't a new memory body inherit the parent's success evidence?
Remembering Is Not Enough: Refining Memories and Learning When to Use Them in Multi-Agent Systems
Show answer and example
Answer: Effects are tied to the body actually used. The new body has not shown the same reliability. Keep identity, owner and compliant correction history, but clear previous effect evidence and mark the new version provisional. This versioning component is not evidence of LLM repair or multi-agent ability.
Illustrative example: Example: v2 worked on ten tasks. If v3 adds a stopping condition, keep v2 records but start v3's effects empty; do not say v3 passed the ten tasks.
Sections in the Chinese source: 核心公式, 数据协议与实现边界, 本地结果与负控. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why compare a proposed repair to both its parent memory and a no-memory baseline?
Remembering Is Not Enough: Refining Memories and Learning When to Use Them in Multi-Agent Systems
Show answer and example
Answer: The parent may already be harmful. Merely beating it can still underperform injecting nothing. For each task in the fixed trigger and history sets, the proposal must match the better reference minus tolerance, without adding outright failures. This protects tested behavior, not every unseen task.
Illustrative example: Example: Parent 0.2, no memory 0.8, proposal 0.7. The proposal beats its parent by 0.5 but misses the 0.79 threshold with default tolerance 0.01; reject it.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
How does testing only on the triggering failure differ from DiscernMem's cross-task validation?
Remembering Is Not Enough: Refining Memories and Learning When to Use Them in Multi-Agent Systems
Show answer and example
Answer: A local fix can harm past capabilities. DiscernMem fixes a set of triggers, previous positive-effect tasks and related tasks, then applies thresholds per task, not to the mean. Related tasks must not regress; they need not improve. Origin tasks are excluded from history validation so the original example cannot repeatedly validate its own memory.
Illustrative example: Example: Trigger improves 0.4→0.9 and an old task drops 0.8→0.5. Reject even though the mean improves.
Sections in the Chinese source: 核心公式, 数据协议与实现边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Why not rewrite every memory used after a team failure?
Remembering Is Not Enough: Refining Memories and Learning When to Use Them in Multi-Agent Systems
Show answer and example
Answer: Another role, a tool or an out-of-scope scenario may have caused the failure. DiscernMem compares replay with just the suspect memory against no-memory replay on the same task, seed and configuration. It then asks whether the rule body is defective or a useful rule was used out of scope. A negative effect alone does not prove a defective body.
Illustrative example: Example: Repair the source before running tests may help with code bugs. If the test environment is broken, more source edits can harm. Add an avoid scope rather than delete a useful source-repair rule.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Core mechanism
Why does ResCom decompose whitened output cross-statistics rather than truncate the original weights?
ResCom: Activation Residual Compensation for Large Language Model Quantization
Show answer and example
Answer: Rank should go to residual directions that occur on calibration data and affect outputs. The Gram matrix measures residual scale and correlations; error–residual cross-statistics measure output impact. Cholesky whitening then truncated SVD solves a ridge-regularized output-reconstruction objective. Direct weight truncation ignores residual distribution and solves a different problem.
Illustrative example: Example: A large residual may have little output effect while a small one is amplified. Allocate rank by output error, not residual size alone.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Implementation details
Why not attach each layer's compensation as soon as its statistics are collected?
ResCom: Activation Residual Compensation for Large Language Model Quantization
Show answer and example
Answer: Early compensation changes the activations at later layers, so statistics no longer describe the same frozen model. Collect every target layer first, then install branches. Streaming merges only data from the same frozen mapping. G and C are sums: averaging without adjusting ridge changes the solution.
Illustrative example: Example: Doubling samples doubles G and C, changing a fixed ridge's relative strength. Dividing them by sample count is a different objective unless ridge is adjusted.
Sections in the Chinese source: 核心公式, 本地复现, 检查与负控. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Method comparison
What changes when a low-rank branch sees rounding residuals rather than quantized input?
ResCom: Activation Residual Compensation for Large Language Model Quantization
Show answer and example
Answer: A branch reading quantized input cannot distinguish two inputs on the same grid point. A residual branch reads the discarded component. ResCom leaves the original path frozen, unlike approaches that remove high-precision weights before quantizing the remainder.
Illustrative example: Example: Removing weights before quantization changes calibration; adding UVd+b beside a frozen path does not change its quantization settings.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Basics
How can a quantized model recover information from the current activation's rounding residual?
ResCom: Activation Residual Compensation for Large Language Model Quantization
Show answer and example
Answer: Quantization maps a range of continuous inputs to one grid value. The forward pass can access both pre-rounding and rounded activations, so their difference retains information the quantized value lost. ResCom reads that residual through a fixed low-rank branch rather than retraining the quantizer.
Illustrative example: Example: 1.01 and 1.04 both round to 1.0. Residuals of 0.01 and 0.04 let compensation differ. This is not a model accuracy result.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Implementation details
Does Meta's release of full RPL code automatically make the local typed-program component a full reproduction?
Learning to Program Retrievals: Composing Search Primitives via Reinforcement Learning
Show answer and example
Answer: No. Release improves provenance, not local fidelity. Local work still omits neural retrievers, cross-turn Python state, policy generation, GRPO, and distillation. The main project's CC-BY-NC and vendored licenses still apply. Public code is not permission to copy or upgrade evidence.
Illustrative example: Example: Having factory blueprints does not mean a small pipe simulator makes the same product. The real execution path and costs still need verification.
Sections in the Chinese source: 论文信息, 背景与主要改动. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Implementation details
The formal paper pools before projection, but the local path pools official last_hidden_state. Is a layer missing?
EmbeddingGemma 2: Unified and Efficient Multimodal Representations
Show answer and example
Answer: No. The pinned official implementation applies a bias-free embedding_projection before returning its 768-dimensional token states. Equal-weight masked mean commutes with linear projection, so pooling those outputs matches the paper; projecting again would be incorrect. The formal paper does not upgrade the existing 16-image GPU run to seven full benchmarks.
Illustrative example: Averaging two tokens after multiplying each by the same matrix equals averaging first and then multiplying. Nonlinearity or token-dependent transformations break this equivalence.
Sections in the Chinese source: 核心公式, 结果与边界. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Core mechanism
Why is an input function times an output vector a rank-one operator?
Show answer and example
Answer: A neuron returns a scalar activation, which only changes the magnitude along one output-weight vector. The input function maps into one output direction, forming a rank-one tensor. Neuron inner products multiply an activation kernel by the output-vector inner product, not input-weight cosine alone.
Illustrative example: Example: Activation 0.2→0.8 changes 0.2w→0.8w, still along w. Changing the output vector changes functional similarity.
Sections in the Chinese source: 核心公式, 原论文关键图. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Implementation details
Why freeze initial parameter savings and adjust downstream BN mean during pruning?
Show answer and example
Answer: The pruning priority divides distortion by initial parameter savings; recomputing from shrinking neighbor shapes changes the criterion. Remove the deleted activation's mean contribution from dense consumers' BN running means to avoid an extra shift. This does not preserve every prediction, and exact undo does not prove lossless accuracy.
Illustrative example: Example: Mean activation 0.8 and output weights [1,2] imply subtracting [0.8,1.6] from the consumer's BN mean, not adding it.
Sections in the Chinese source: 核心公式, 数据协议、结果与负控, 高成本完整实验与复现边界. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Method comparison
Is HOPE Hilbert compression the same as the Hope network in Sleep?
Show answer and example
Answer: No. Hilbert Operator for Progressive Encoding analyzes frozen-network functional capacity and compresses with PRUNE, MERGE and EVICT. Sleep's Hope is a multi-frequency memory network updated in learning and offline consolidation. Local coverage is the former's kernel, pruning and one-step restoration components.
Illustrative example: Example: Scoring a frozen ResNet's neurons for removal is not teaching a language model during sleep.
Sections in the Chinese source: 背景与主要改动, 保留机制与接口, 高成本完整实验与复现边界. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Basics
Why doesn't HOPE simply remove neurons with small weights?
Show answer and example
Answer: Scaling a ReLU neuron's input and BN scale up and its output weights down can leave the function unchanged. Raw weight magnitude is therefore unreliable. HOPE compares Hilbert-operator capacity from activation functions and output vectors under a Gaussian surrogate, not proven importance on real data.
Illustrative example: Example: Multiply the hidden activation by seven and divide output weights by seven. Predictions and functional capacity stay unchanged, although weight magnitude falls.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Core mechanism
Does LTI's edit-distance threshold cap edit count or a similarity ratio?
Language Models Need Sleep: Learning to Self-Modify and Consolidate Memories
Show answer and example
Answer: It caps the absolute number of token edits, inclusively. For accepted pairs the literal reward is one minus distance divided by the longer completion length. A fixed similarity-ratio threshold differs.
Illustrative example: Example: Threshold 1 admits one substitution in both 6- and 60-token completions, with rewards 5/6 and 59/60.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Implementation details
Can the local Sleep reward prove the model retains knowledge long term?
Language Models Need Sleep: Learning to Self-Modify and Consolidate Memories
Show answer and example
Answer: No. It receives tokenized completions and external semantic labels and checks reward boundaries. No dream sampling, CMS, expansion, LoRA, screening or forgetting evaluation. The costly full experiment is not reproduced; the kernel is not an Evolve capability operator.
Illustrative example: Example: Reward 0.9 for two integer sequences confirms the formula for that input, not any update to model weights.
Sections in the Chinese source: 本地复现与边界, 高成本完整实验. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Method comparison
Does crossing the literal-distance threshold also zero semantic reward?
Language Models Need Sleep: Learning to Self-Modify and Consolidate Memories
Show answer and example
Answer: No. A separate frozen judge assigns binary semantic reward. The threshold only gates the literal branch. Very different wording can preserve meaning; replacing the judge with edit distance changes the method.
Illustrative example: Example: With gamma 0.4 and a positive semantic judgment, a zero literal branch still yields total reward 0.4.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Basics
Is Sleep just a pause or a separate learning stage?
Language Models Need Sleep: Learning to Self-Modify and Consolidate Memories
Show answer and example
Answer: It is offline consolidation. Before applying precomputed base updates, expand slow modules and use the pre-update teacher to consolidate fast memories into the student to be updated. Then apply the update and clear sending-layer experts. It is not just a pause or brief replay, and the full chain is not implemented locally.
Illustrative example: Example: Verify notes copied into a long-term library before deleting the temporary notes. Computing note similarity is not consolidation.
Sections in the Chinese source: 背景与主要改动, 高成本完整实验. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Core mechanism
Why does HGE separately predict container conversion, conditional item purchases, and quantities instead of one store GMV?
Show answer and example
Answer: Aggregate GMV is sparse and noisy. Decompose transaction probability and conditional item contributions, with residuals for top-K omissions. Items are not mutually exclusive, and the fusion gate uses context rather than realized conversion.
Illustrative example: A shopper can buy two inexpensive items or one expensive item; one categorical choice cannot represent both patterns.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Implementation details
Why must HGE not select residual and fusion branches using realized conversion?
Show answer and example
Answer: Residuals can correct over- and underestimation and need not be nonnegative. Predict gates from pre-impression context, not observed transaction labels. The component lacks full multitask training, and differentiating squared output is a gradient diagnostic only.
Illustrative example: Using the decomposition only after learning a store converted leaks the answer and inflates test results.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Method comparison
Why do HGE item sigmoid and mutually exclusive softmax imply different business assumptions?
Show answer and example
Answer: A transaction can contain multiple items, so independent sigmoid scores can all be high. Softmax forces one distribution with a fixed sum and changes basket semantics. Quantity is a distinct head.
Illustrative example: Buying coffee and bread together should not require the probability of bread to fall when coffee increases.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Basics
Are HGE's top-K items the displayed recommendations or inputs within one container?
Show answer and example
Answer: They are item inputs for each candidate store or room. Set attention models internal relations and combines user/query context to score the container. Top-K is partial inventory, not a list of K stores or item outputs.
Illustrative example: Three internal products from each of two stores yield six encoded items but only two store scores.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Core mechanism
Why reject as easy only unanimous matches to the verified candidate answer?
On-Policy Self-Training: Data-Free Self-Improvement in Large Language Models
Show answer and example
Answer: Agreement alone doesn't show mastery. All students can agree on the wrong answer. Privileged clues only guide teacher scoring on student prefixes; self-generated problems need independent re-answering and meaning-preserving checks.
Illustrative example: A unanimous wrong number remains useful until compared with the verified candidate answer.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Implementation details
Is OPST EMA β old-teacher momentum or new-student weight?
On-Policy Self-Training: Data-Free Self-Improvement in Large Language Models
Show answer and example
Answer: It is new-student weight: after each student update, use (1−β) old teacher + β new student. Normalize per response then average equally. Document formula ambiguity instead of claiming unquestioned exactness.
Illustrative example: Small β means slowly absorb the student, not almost replace the teacher.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Method comparison
How does OPST easy filtering differ from removing student consensus?
On-Policy Self-Training: Data-Free Self-Improvement in Large Language Models
Show answer and example
Answer: Only unanimous matches to the candidate answer are too easy. Unanimous disagreement may signal a gap or bad problem. Removing all consensus changes the gate.
Illustrative example: Identical wrong results don't establish mastery.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Basics
Why does OPST generate both problems and privileged clues?
On-Policy Self-Training: Data-Free Self-Improvement in Large Language Models
Show answer and example
Answer: Reduce reliance on human problem banks and give teacher extra solving context while students generate from prompts only. Validate generated tasks through independent re-answering, meaning-preserving rewrites and difficulty filtering.
Illustrative example: Resolve a generated problem independently to reject inconsistent prompts or answers.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Core mechanism
Why keep original advantages and denominator after masking penalties beyond the last unrecovered error?
When to Stop Penalizing Actions? Martingale GRPO for VLA Reinforcement Learning
Show answer and example
Answer: Masking isolates where failure credit applies. Changing group rewards/advantages or denominator adds other effects. Execute whole rollout before VLM hindsight; that does not guarantee stopping-time theory applies.
Illustrative example: Mask idle behavior after a mishap without amplifying punishment for the remaining actions.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Implementation details
May MART recompute advantages or renormalize by unmasked tokens?
When to Stop Penalizing Actions? Martingale GRPO for VLA Reinforcement Learning
Show answer and example
Answer: No. Keep original group advantages and valid-action denominator; apply the mask only to feedback. Valid interval includes its upper endpoint. Invalid format/identity/bounds are errors, not semantic-uncertainty fallback.
Illustrative example: Shrinking the denominator after masking amplifies retained gradients and changes the method.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Method comparison
Why isn't a retrospective VLM boundary automatically a theoretical stopping time?
When to Stop Penalizing Actions? Martingale GRPO for VLA Reinforcement Learning
Show answer and example
Answer: The VLM sees future trajectory, unlike a decision based only on the past. Arbitrary masks don't inherit unbiasedness or variance reduction guarantees.
Illustrative example: Choosing an important step after seeing the outcome differs from deciding as it happens.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Basics
Why mask late feedback on MART-GRPO failures?
When to Stop Penalizing Actions? Martingale GRPO for VLA Reinforcement Learning
Show answer and example
Answer: Terminal failure doesn't mean every later action caused it. Find the last unrecovered error and mask subsequent negative credit while retaining relevant earlier signal; execution itself is not stopped early.
Illustrative example: A failed regrasp after a drop needn't be punished equally solely because the episode failed.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Core mechanism
Why does adding log(Nref/Na) to action logits cancel identical action-token duplication?
UniAction: Unified Action Conditioning for World Models
Show answer and example
Answer: In joint softmax, copying identical keys, values, and positions increases the action exponent sum. The log-count bias cancels that multiplicity while preserving relative action logits. Different new content or positions can still affect outputs.
Illustrative example: Duplicating each token with identical position would raise uncorrected action weight; corrected output should not change from count alone.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Implementation details
Does duplication invariance allow arbitrary added actions or changed positions?
UniAction: Unified Action Conditioning for World Models
Show answer and example
Answer: No. Exact invariance requires identical key, value, and position representations. Na counts valid actions per request; padding is excluded and has zero attention. Supply the native position callback explicitly; local diagnostics are not Wan's native position mechanism.
Illustrative example: Copying the same action at another time changes position, so changed output does not violate invariance.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Method comparison
How does corrected joint softmax differ from separately normalizing video and action then summing?
UniAction: Unified Action Conditioning for World Models
Show answer and example
Answer: Joint normalization shares one denominator and applies log(Nref/Na) to action logits. Separate softmax gives each stream total mass one before merging, a different cross-modal weighting even if internal action ranks match.
Illustrative example: Strong video and weak action evidence can yield low total action weight jointly, but separate softmax forces a full action share.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Basics
Why should action token count not directly represent importance across formats?
UniAction: Unified Action Conditioning for World Models
Show answer and example
Answer: Equivalent actions can be tokenized at different granularity. UniAction separates element identity and position and corrects joint attention for count effects. Genuine added actions still carry new information; not every longer encoding is meaningless duplication.
Illustrative example: Splitting a path into 20 rather than ten segments does not double importance, but adding a real grasp matters.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Core mechanism
Why does FlexGlue read coordinates near a peak rather than take a global expectation?
FlexGlue: Sparse-to-Dense Feature Matching and Visual Disambiguation
Show answer and example
Answer: Similar structures can create several peaks, making a global average land between valid matches. A local window around the strongest candidate avoids averaging modes; refinement then predicts a displacement. Covisibility votes and hard negatives separately assess whether the images are matchable.
Illustrative example: If two identical windows score highly, the global average lands on the wall. Read coordinates around the chosen window for a plausible starting point.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Implementation details
Can ordinary softmax and cross-entropy reproduce FlexGlue's coarse matching?
FlexGlue: Sparse-to-Dense Feature Matching and Visual Disambiguation
Show answer and example
Answer: No. The paper uses 1.5-entmax and Fenchel–Young loss, with bilinear weights on four cells surrounding the subpixel target and detached coarse coordinates during refinement. No full local operator, training metrics, or GPU receipt exists.
Illustrative example: A target at a four-cell junction gives all four weights. Snapping to one cell changes the supervision.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Method comparison
When does FlexGlue's layer-wise covisibility gate exit early?
FlexGlue: Sparse-to-Dense Feature Matching and Visual Disambiguation
Show answer and example
Answer: Every layer produces token-level sigmoid votes. Exit when the mean across both images falls below that layer's threshold tγ^(L−k). This tests whether matching should continue, not just the strongest local match. Reliability of the gate matters; fewer layers do not prove higher quality.
Illustrative example: Two symmetric buildings can contain nearly identical local features but have low overall covisibility, so exit rather than refine a confident false match.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Basics
How does FlexGlue's dense grid support arbitrary-position queries?
FlexGlue: Sparse-to-Dense Feature Matching and Visual Disambiguation
Show answer and example
Answer: Features interact globally on a stride-8 grid. Interpolation reads the requested coordinate's feature, and cross-image attention estimates the corresponding location. Queries need not coincide with cell centers or be restricted to fixed keypoints.
Illustrative example: For a query between four cells, interpolate its feature before matching instead of snapping to the nearest cell.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Structured Decisions / Typed Readout: From Generated Sentences to Candidate Distributions
Core mechanism
Why isn't PathJev just softmax on pathology images?
PathJev: Decision-Native Vision–Language Modelling for Histopathology
Show answer and example
Answer: Its head combines candidate-linear terms, question interactions and set attention, inherited from earlier work. New evidence depends on pathology-state data, typed expert labels, overlap audit and unseen organs. A small classifier does not prove pathology transfer. Full training was not reproduced; targets stay out of prompts.
Illustrative example: Example: Tumor grades are text candidates scored at runtime and trained with source labels. A softmax gradient alone doesn't validate cross-organ recognition.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Structured Decisions / Typed Readout: From Generated Sentences to Candidate Distributions
Implementation details
Can PathJev replace the Score target with an ordinal-distance loss?
PathJev: Decision-Native Vision–Language Modelling for Histopathology
Show answer and example
Answer: Not within the same protocol. The paper uses one-hot grade supervision; distances define a new objective even for ordered labels. Targets never enter prompts, and full training was too costly and not reproduced.
Illustrative example: Example: Grades two and three are adjacent, but changing their penalty is not reproducing the original objective.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Structured Decisions / Typed Readout: From Generated Sentences to Candidate Distributions
Method comparison
How does PathJev's unseen-organ test differ from random holdout of seen organs?
PathJev: Decision-Native Vision–Language Modelling for Histopathology
Show answer and example
Answer: Unseen organs test cross-organ transfer. Random holdout mostly tests new samples from familiar distributions. Audit overlap to keep related states out of training. The same inherited head does not make the splits equally hard or equivalent.
Illustrative example: Example: Unseen slices of organ A after training on A are not transfer to organ B.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Structured Decisions / Typed Readout: From Generated Sentences to Candidate Distributions
Basics
Why aren't PathJev typed decisions open-ended pathology QA?
PathJev: Decision-Native Vision–Language Modelling for Histopathology
Show answer and example
Answer: Expert labels become structured choices or scores normalized over runtime candidates. This helps evaluation but does not cover all open-ended pathology explanations.
Illustrative example: Example: Choosing a predefined grade is different from writing a diagnostic report.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Structured Decisions / Typed Readout: From Generated Sentences to Candidate Distributions
Core mechanism
Why run every editor offline but only the selected one at inference?
EditJev: System One Routing for Image Editing Models
Show answer and example
Answer: All offline outputs and their judgments supply pairwise quality labels. At deployment, predicted utility plus fixed speed preference selects once without seeing outputs. Candidate ordering must match; speed weighting is a tradeoff, not truth. The full multi-editor setup is too costly and was not reproduced.
Illustrative example: Example: For background removal, pick an editor first and call only it. Running every editor defeats the cost savings.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Structured Decisions / Typed Readout: From Generated Sentences to Candidate Distributions
Implementation details
Why normalize EditJev's pairwise loss by valid pairs per request first?
EditJev: System One Routing for Image Editing Models
Show answer and example
Answer: Requests have different counts of reliable pairs. Average within request and then across requests to prevent pair-rich requests dominating. A request with no valid pair contributes differentiable zero.
Illustrative example: Example: An image with many easily distinguished outputs shouldn't outweigh all others simply because more pairs can be formed.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Structured Decisions / Typed Readout: From Generated Sentences to Candidate Distributions
Method comparison
How can EditJev express quality-first and speed-first choices in one interface?
EditJev: System One Routing for Image Editing Models
Show answer and example
Answer: Combine quality utility with a fixed speed score weighted by beta. Changing beta changes the choice, not the quality labels. It expresses a cost preference, not a universally best setting without evidence.
Illustrative example: Example: If quality is close, weighting speed more can select the faster editor without making it higher quality.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Structured Decisions / Typed Readout: From Generated Sentences to Candidate Distributions
Basics
Does EditJev predict the edited image?
EditJev: System One Routing for Image Editing Models
Show answer and example
Answer: No. It predicts an expected-utility vector for fixed editors and routes the request to one. That editor produces the image; router supervision comes from judging offline outputs across editors.
Illustrative example: Example: For a clothing recolor, the router picks an editor rather than painting the result.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Core mechanism
Why predict future latents from history and expose true future only to the training target encoder?
DT-JEPA: Learning to Predict Future Representations for Decision-Making
Show answer and example
Answer: The future is unavailable at decision time, so reading it would leak. History predicts a distribution and reparameterized samples condition decisions. The EMA target only supplies training alignment. Future representations distinguish dynamics even at equal return.
Illustrative example: Two trajectories can score ten, yet one hits an obstacle and the other stays clear.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Implementation details
Can NLL, BC, and critic reductions all use generic MSE.mean?
DT-JEPA: Learning to Predict Future Representations for Decision-Making
Show answer and example
Answer: No. Sum NLL over latent dimensions before the batch mean; sum BC and critic over time and action dimensions before the batch mean. Reductions change relative weights. Detach target and TD bootstrap, but retain actor gradients through Q with respect to actions. Local tests omit full online data collection.
Illustrative example: Doubling action width changes an all-element mean's per-example contribution relative to other summed terms.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Method comparison
What is the trade-off between a Gaussian future distribution and a deterministic vector?
DT-JEPA: Learning to Predict Future Representations for Decision-Making
Show answer and example
Answer: Mean and variance express uncertainty; reparameterized samples condition DT. A deterministic vector is a single point. The diagonal Gaussian is still limited and does not cover arbitrary multimodality or allow real future input at deployment.
Illustrative example: Variance can represent uncertainty between routes, but a single Gaussian may not capture two distant discrete destinations.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Basics
Why might a Decision Transformer need more than expected return?
DT-JEPA: Learning to Predict Future Representations for Decision-Making
Show answer and example
Answer: Equal returns can lead to different states, so one return-to-go cannot distinguish dynamics. DT-JEPA conditions actions on a predicted future representation distribution from history, not an observed future. Manage uncertainty and isolate training-only inputs.
Illustrative example: Routes with equal reward may reach a charger or an exit; state predictions provide information absent from the scalar.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Core mechanism
Why hide final metrics from the inner agent while letting the outer loop evaluate strategy with them?
RSI-Opt: Recursive Self-Improvement Optimizer Agents For Model Training
Show answer and example
Answer: The inner agent edits an updater from training loss and diagnostics. Final metrics would let it tune directly against evaluation. The outer loop compares fixed strategy outcomes and has a different selection role. Inherited prompts, libraries, and cadence make it recursive; normalized rankings are not unseen-task generalization.
Illustrative example: Example: Tune update rules from training loss; compare strategies after execution rather than inspect test scores for every edit.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Implementation details
What are the evidence limits of archiving low-score descendants and removing exposed branches?
RSI-Opt: Recursive Self-Improvement Optimizer Agents For Model Training
Show answer and example
Answer: Archive evaluated low-score strategies to preserve failed exploration rather than fake monotonic progress. Removing a branch exposed to held-out descriptions may not remove summaries read by other descendants. The paper leaves that provenance issue open. Z-score aggregation is not significance testing.
Illustrative example: Example: Deleting a contaminated policy while retaining summaries based on it can still leak to descendants.
Sections in the Chinese source: 核心公式, 本地复现. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Method comparison
Why fix model, data order, and update budget across candidates rather than only seed?
RSI-Opt: Recursive Self-Improvement Optimizer Agents For Model Training
Show answer and example
Answer: Changing architecture, loss, or duration confounds updater effects. A fixed contract controls the optimization problem, and failed updater trials can roll back. Multiple seeds do not repair systematic protocol differences or make training-suite scores generalization.
Illustrative example: Example: A gets ten thousand steps and B gets a thousand. The same seed does not attribute their score difference to the updater.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Basics
Why does RSI-Opt improve optimization methods instead of only optimizing one fixed model?
RSI-Opt: Recursive Self-Improvement Optimizer Agents For Model Training
Show answer and example
Answer: An update that works now may fail on other models or stages. RSI-Opt treats the updater as the evolving object, evaluates controlled runs, and decides what to keep. Distinguish inner optimization from outer method selection. Candidate prose or one strong score does not establish broad gains.
Illustrative example: Example: Train a small model with one rule, then compare rules under equal budgets rather than retain one run's best weights.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Core mechanism
Why can't a high in-loop image score automatically activate an extracted strategy?
ImageRSI: Verification-Driven Recursive Self-Improvement for Agentic Image Generation
Show answer and example
Answer: A judge that guides repairs and scores their gains can amplify its biases. Compare with and without experience at matching request, configuration, and seed; calibrate with independent blinded judgments and admit on a held-out set. Hard constraints cannot be averaged away. Returning a fallback is not acceptance.
Illustrative example: Example: A strategy improves composition to please the judge but misses the required person count. Independent matched tests and vetoes expose this.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Implementation details
Does returning the highest-scoring fallback after two repairs mean accepted?
ImageRSI: Verification-Driven Recursive Self-Improvement for Agentic Image Generation
Show answer and example
Answer: No. The best of the initial image and up to two repairs may be returned even if all fail hard constraints. Judge SFT/GRPO updates between collection rounds, not within one evaluation request. Local generation, semantic judging, and training are unimplemented, so formulas are not system verification.
Illustrative example: Example: If all three images misspell required text, the prettiest fallback still fails the checklist.
Sections in the Chinese source: 核心公式, 本地复现. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Method comparison
Why separate experience extraction from independent paired admission?
ImageRSI: Verification-Driven Recursive Self-Improvement for Agentic Image Generation
Show answer and example
Answer: A strategy inferred from a trajectory is only a candidate. Compare with and without it under matched request, input, configuration, and seed, then blind-calibrate and use independent holdout. In-loop judges may favor their own guidance; test verdicts must not guide generation.
Illustrative example: Example: Stronger spatial wording works on one image. Pair on new requests to see if it harms other relations or styles before admitting it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Basics
Why do fixed original requirements matter more than increasing a five-axis average?
ImageRSI: Verification-Driven Recursive Self-Improvement for Agentic Image Generation
Show answer and example
Answer: Repair must meet original counts, relationships, wording, and identity constraints. It cannot discard difficult conditions or average away failures. Five dimensions diagnose issues; hard vetoes govern acceptance. A high score does not guarantee image facts.
Illustrative example: Example: Beauty cannot compensate for two cups when three and correct text were required. Do not change the request to two or three.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Core mechanism
Why do closed arguments and true guards not prove a skipped model call was correct and actually avoided?
HarnessMM: Recursive Self-Improvement for Harness Token Efficiency
Show answer and example
Answer: They show an action can be reused, not that it completes the task; the original evaluator still judges semantics. Execution certificates must bind exact arguments, predecessor receipts, and tool outputs, and audit that the registered slow call did not occur. Theoretical savings differ from actual calls; include failures and recovery.
Illustrative example: Example: Directly run a saved test command, keep the receipt, and verify no extra slow model request before counting a saving.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Implementation details
Why include failures, fallbacks, and development in HarnessMM cost accounting?
HarnessMM: Recursive Self-Improvement for Harness Token Efficiency
Show answer and example
Answer: An episode counts all real input and output tokens: structure generation, selection, extra context, recovery, and final synthesis, including cached input. Record development separately and amortize over deployments. One fewer slow call does not guarantee lower total tokens, latency, or energy.
Illustrative example: Example: Saving one main-model call but adding two selection/recovery calls requires counting all. Amortize graph construction over actual reuse.
Sections in the Chinese source: 核心公式, 本地复现. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Method comparison
How does reusing a fixed action differ in online freedom from planning a new action?
HarnessMM: Recursive Self-Improvement for Harness Token Efficiency
Show answer and example
Answer: Reuse is limited to previously resolved actions with complete arguments and satisfied guards. It cannot invent steps or fill parameters from new evidence. Planning interprets observations and constructs actions. Reuse saves repetition only where covered; otherwise abstain. Structural success is not semantic correctness.
Illustrative example: Example: A saved test command can trigger on success. A new dependency conflict cannot be fixed by guessing a command in the reuser.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Basics
Why can another model call after tool feedback repeat an already completed decision in HarnessMM?
HarnessMM: Recursive Self-Improvement for Harness Token Efficiency
Show answer and example
Answer: The prior turn can decide a later command while waiting for a prerequisite. Once a guard holds, another reasoning call may add nothing. HarnessMM reuses committed decisions to avoid slow calls, but genuinely new interpretation goes back to the model. Reuse is not arbitrary future prediction.
Illustrative example: Example: If a specified test should follow successful file creation, trigger it from the receipt. An unexpected error needs new reasoning.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
Why average student–teacher disagreement within trajectories and then across trajectories instead of across node visits?
Frontier-OPD: Discovering the Edge of Agent Capability and Teaching Agents to Go Beyond
Show answer and example
Answer: Loops repeatedly visit states; counting each visit as independent evidence lets one circling path dominate. Two-stage averaging balances trajectory weight. Representative histories then support real teacher-bridge trials. Signed disagreement for ranking differs from agreement for validation; a high node score does not guarantee takeover helps.
Illustrative example: Three visits at disagreement ten in one trajectory and one zero in another give a balanced score of five. Visit averaging gives 7.5 and overweights the loop.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
How are windows compared when the bridge is shorter than K, and can its continuation enter training?
Frontier-OPD: Discovering the Edge of Agent Capability and Teaching Agents to Go Beyond
Show answer and example
Answer: Compare equal windows of actual bridge length plus H. Require valid replay, nonterminal handoff, and complete continuation; accept bridge responses only when agreement does not decrease. Student continuation used for validation cannot enter training, and retain repeated responses separately. The local path is only the L1 numeric protocol, without the trainer or full experience pool.
Illustrative example: If K caps the bridge at two turns but the teacher uses one, compare windows of one plus H. Do not use a longer historical window or train on the validation continuation as teacher demonstration.
Sections in the Chinese source: 核心公式, 本地复现. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
Why use different criteria for high-ranking frontier nodes and valid teacher bridges?
Frontier-OPD: Discovering the Edge of Agent Capability and Teaching Agents to Go Beyond
Show answer and example
Answer: Signed teacher–student disagreement prioritizes where to spend limited teacher budget. Bridge validation compares actual takeover plus student continuation to an equally long historical window, requiring valid replay and nondecreasing agreement. Prioritization is not training-data admission; a high-ranked node does not prove teaching works.
Illustrative example: High disagreement may warrant a trial even if the student still cannot continue after takeover. Do not add that failed bridge to training merely because it ranked high.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why does Frontier-OPD ask teachers to take over only at selected states instead of generating all training trajectories?
Frontier-OPD: Discovering the Edge of Agent Capability and Teaching Agents to Go Beyond
Show answer and example
Answer: Full teacher interaction is expensive, and imitating smooth teacher trajectories may miss student bottlenecks. Frontier-OPD finds teachable frontiers from student-visited states, briefly bridges with the teacher, then hands control back. Savings must be accompanied by verifying that the student can continue after the bridge.
Illustrative example: If the student repeatedly gets stuck opening a door after finding its key, test a short teacher intervention there rather than having the teacher finish every task from scratch.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Core mechanism
Why isn't PAIR's multi-negative preference loss token-level DPO?
PAIR: Preference-Aligned Intent Retrieval for Premium People Recommendation at LinkedIn
Show answer and example
Answer: Clicked candidates, non-clicked candidates from the same exposure, and random members train profile and history encoders for complementary relevance. This is a multi-negative contrastive score objective with no token probability ratios, explicit reference, or KL term. The preference comes from exposure-conditioned supervision.
Illustrative example: Illustration: A job seeker selects a recruiter over a similar job seeker; compare candidate representations, not answer-token probabilities.
Sections in the Chinese source: 背景与主要改动, 核心公式与服务边界. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Implementation details
How verify PAIR history positions, padding, and candidate identity without future clicks?
PAIR: Preference-Aligned Intent Retrieval for Premium People Recommendation at LinkedIn
Show answer and example
Answer: Use only pre-request events, ensure masked padding changes cannot affect scores, and align candidate profiles with labels. Position changes may affect results as designed. Gradient checks and CUDA smoke verify operators, not professional recommendation gains. Do not add a reference KL to the contrastive objective.
Illustrative example: Illustration: Perturbing masked history should not change a score; reading a future clicked candidate to improve prediction leaks the answer.
Sections in the Chinese source: 背景与主要改动, 核心公式与服务边界. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Method comparison
What difficulty do same-exposure non-clicks and random member negatives add in PAIR?
PAIR: Preference-Aligned Intent Retrieval for Premium People Recommendation at LinkedIn
Show answer and example
Answer: Exposed non-clicks represent real competitors; random members offer broad, often easier contrast. The mix learns fine choice and global structure. Non-clicks can reflect position or opportunity, not true negatives. Preference does not imply token DPO or removal of exposure bias.
Illustrative example: Illustration: Two local recruiters create a fine comparison; a random unrelated student is easier and does not replace the exposed choice.
Sections in the Chinese source: 背景与主要改动, 核心公式与服务边界. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Basics
Why does PAIR model professional relevance as complementarity rather than mere similarity?
PAIR: Preference-Aligned Intent Retrieval for Premium People Recommendation at LinkedIn
Show answer and example
Answer: A member may need a recruiter, client, or different skill set. Shared profile encoding and history use clicked versus skipped members under the same exposure to learn such relationships. Complementarity remains scenario-dependent, not a rule to recommend dissimilar people.
Illustrative example: Illustration: An engineer looking for work may need a recruiter; when looking for a technical peer, another engineer may be relevant.
Sections in the Chinese source: 背景与主要改动, 核心公式与服务边界. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why can OneLatent's latent tokens save online decoding without constituting a strict reasoning bottleneck?
OneLatent: Latent Reasoning for Efficient Foundation Recommendation Models
Show answer and example
Answer: Training reconstructs exploration, evaluation, and attribution through latent tokens, then outputs semantic IDs instead of long reasons. Ordinary causal masks still expose original profile/history and all student weights update. Gains cannot be attributed to tokens alone storing every step. Local work reviewed evidence and cost without running a reproduction.
Illustrative example: Illustration: A few internal markers can support a recommendation during training and only an ID appears online; faster output does not prove all reasoning is stored in markers.
Sections in the Chinese source: 背景与主要改动, 成本判断与本地边界. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Why average OneLatent reconstruction over each rationale's tokens, then over available perspectives?
OneLatent: Latent Reasoning for Efficient Foundation Recommendation Models
Show answer and example
Answer: Summation overweights long reasons and examples with more perspectives. Two-level averaging controls length and missingness. Teachers can read positives to create reasons, but the student prefix must exclude the target SID and positive description. Async serving caches and training costs are separate; local work is review-only.
Illustrative example: Illustration: A 200-word rationale should not dominate a 20-word one by length alone, and a missing attribution view is not a real empty-text target.
Sections in the Chinese source: 背景与主要改动, 成本判断与本地边界. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
Why is OneLatent's interface not latent MSE distillation with an extra encoder/projector?
OneLatent: Latent Reasoning for Efficient Foundation Recommendation Models
Show answer and example
Answer: OneLatent trains all parameters of a causal student to reconstruct single- or multi-view reasons and predict SIDs, with no extra projector or strict bottleneck. MSE aligns feature coordinates, not generation. A fixture replacing the objective would be a new diagnostic; no such substitute was run locally.
Illustrative example: Illustration: Reconstructing reasons from markers supervises generatable content; shrinking Euclidean distance between vectors does not demonstrate that ability.
Sections in the Chinese source: 背景与主要改动, 成本判断与本地边界. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Why supervise OneLatent with exploration, evaluation, and attribution instead of one fixed long chain of thought?
OneLatent: Latent Reasoning for Efficient Foundation Recommendation Models
Show answer and example
Answer: The views cover discovery, comparison, and explanation; available perspectives per sample train a latent interface without a rigid template. It reduces explicit serving output, not proof of faithful reasoning recovery. Full-parameter training also contributes.
Illustrative example: Illustration: One example may have only a reliable evaluation, another all three views. They need not be forced into the same reasoning length.
Sections in the Chinese source: 背景与主要改动, 成本判断与本地边界. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Core mechanism
Why give only teacher abstract intent and bound it token by token?
UserArena: Benchmarking Interactive User Simulation for Long-Horizon Shopping Agents
Show answer and example
Answer: Whole trajectories leak future products/actions. Filter abstract preferences for teacher only, preserving student deployment input. Compare teacher views on the same prefix, constrain added guidance and ground on actual responses. KL can't replace leakage checks.
Illustrative example: Durability preference may be available; next item IDs and purchase outcomes may not.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Implementation details
May OPTD grounding replace gold tokens on a student rollout prefix?
UserArena: Benchmarking Interactive User Simulation for Long-Horizon Shopping Agents
Show answer and example
Answer: No. Grounding uses separate true-prefix logits; teacher views score identical student prefixes. Use feasible lower endpoint per token with KL from mixed to no-intent teacher, not batch-mean reverse KL.
Illustrative example: Injecting the actual next action after a divergent student history is not teacher forcing on a real trajectory.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Method comparison
How does OPTD teacher intent differ from giving the student future trajectories?
UserArena: Benchmarking Interactive User Simulation for Long-Horizon Shopping Agents
Show answer and example
Answer: Only frozen teacher sees abstract intent, excluding concrete action/item identity, price and outcome. Student continues its own prefixes. Trust regions bound effects but don't prove no extraction leakage.
Illustrative example: Lightweight-item preference is allowed, next clicked product is not.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Basics
Why does UserArena report more than next-action accuracy?
UserArena: Benchmarking Interactive User Simulation for Long-Horizon Shopping Agents
Show answer and example
Answer: Simulation should preserve trajectory dynamics, cross-session preferences and cohort differences, and reproduce differences between recommendation policies. Local hits aren't whole-user fidelity.
Illustrative example: Good click prediction with wrong long-run interest drift can still distort policy evaluation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
Why is a student continuation after an expert prefix still off-policy?
MOER: Mix-Policy Optimization under Expert Prefix Guidance with Selective Rollback
Show answer and example
Answer: It is conditional on a supplied prefix, not a complete sample from unguided student. MOER keeps successful guided traces and falls back to verified expert traces on failure, using distinct objectives. Expert data stay in training.
Illustrative example: Completing a proof after the teacher provides the key step is not reaching that state independently.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
How can MOER's off-policy ratio equal one forward yet have gradients?
MOER: Mix-Policy Optimization under Expert Prefix Guidance with Selective Rollback
Show answer and example
Answer: The denominator is detached and numerator differentiable, so equal values do not cancel derivatives. Joint group advantages are not divided by SD; normalize over all valid response tokens, not extra mixture-weighted counts.
Illustrative example: A numerator divided by its detached copy is one, but differentiating numerator still changes loss.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
How does MOER distinguish native on-policy and guided trajectories?
MOER: Mix-Policy Optimization under Expert Prefix Guidance with Selective Rollback
Show answer and example
Answer: The whole guided trace is off-policy even if the student wrote the suffix, because its starting state was externally influenced. Use different objectives, not token authorship to reclassify the suffix.
Illustrative example: A student writing step three after two expert steps remains in the guided trace.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
Why progressively shorten MOER expert prefixes?
MOER: Mix-Policy Optimization under Expert Prefix Guidance with Selective Rollback
Show answer and example
Answer: Complete solutions limit exploration, but no guidance can be too hard. Shortening transfers more continuation to the student. If continuation fails, roll back to a verified complete expert trace rather than treating failure as demonstration.
Illustrative example: Give two steps, then one, leaving more of the derivation to the student.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Core mechanism
Why can't gradient-based GIFT selection promise permanent absence of conflict?
GIFT: Gradient Informed Filtering for Distillation Augmented Reasoning RL
Show answer and example
Answer: Prioritize nonconflicting and orthogonally informative tokens; quotas can require weaker conflicts. Selection isn't projection, and normalize by all valid tokens. Local logit geometry doesn't guarantee finite shared-parameter updates.
Illustrative example: A subbatch quota larger than conflict-free availability can select the least conflicting extras.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Implementation details
Can GIFT renormalize by selected token count?
GIFT: Gradient Informed Filtering for Distillation Augmented Reasoning RL
Show answer and example
Answer: No. Divide by all valid tokens. Quotas depend on subbatch valid/eligible sets. Follow explicit handling for missing RL and zero teacher gradients rather than automatically treating them as novel directions.
Illustrative example: Selecting half and dividing by half doubles retained contribution.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Method comparison
How does GIFT token selection differ from projecting teacher gradients?
GIFT: Gradient Informed Filtering for Distillation Augmented Reasoning RL
Show answer and example
Answer: Selected tokens retain original distillation losses; gradients aren't rotated. Exact quotas may admit weak conflicts, so there is no universal finite-step safety guarantee.
Illustrative example: An accepted position uses its original loss, not a geometrically replaced gradient.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Basics
Why inspect RL and distillation gradient geometry in GIFT?
GIFT: Gradient Informed Filtering for Distillation Augmented Reasoning RL
Show answer and example
Answer: Teacher signal can conflict or add an orthogonal direction. Select by local logit gradients to reduce conflict and narrow supervision rather than imitate every token.
Illustrative example: Teacher format preferences can oppose task reward, so inspect the directions.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Core mechanism
Why do both AOPD terms share a truncated teacher measure?
When to Follow the Teacher: Advantage-Based On-Policy Distillation
Show answer and example
Answer: The estimator combines terms that cancel to measure teacher-action advantage for student continuation. Clipping only one or renormalizing changes the objective. Use other same-prompt rollouts for LOO, with frozen teacher/behavior and no new counterfactual continuations.
Illustrative example: Evaluate a teacher-preferred opening against the student's actual returns rather than reinforce it solely from teacher likelihood.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Implementation details
Can AOPD clip only the sampled term or replace rollout S with current student?
When to Follow the Teacher: Advantage-Based On-Policy Distillation
Show answer and example
Answer: No. Both terms share the unnormalized truncated teacher measure; denominator is frozen rollout S. Update only Sθ, with groups larger than one for independent LOO. Don't change behavior probabilities across steps.
Illustrative example: Re-softmax after truncation changes total mass and correspondence between terms.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Method comparison
How does AOPD differ from multiplying KL by reward?
When to Follow the Teacher: Advantage-Based On-Policy Distillation
Show answer and example
Answer: It includes a sampled term and full-distribution term under the same unnormalized truncated teacher measure, plus independent leave-one-out returns. Simple reward multiplication loses that estimator structure.
Illustrative example: Weighting one sampled word doesn't reproduce subtraction of the vocabulary-wide term.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Basics
Why ask whether the student can complete an AOPD teacher action?
When to Follow the Teacher: Advantage-Based On-Policy Distillation
Show answer and example
Answer: Teacher's favored next step may be beyond student continuation skill. Relative terminal advantage measures usable guidance from existing student rollouts without counterfactual generations.
Illustrative example: An advanced proof start is not useful if the student cannot continue it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Core mechanism
Why normalize TTR keys but preserve target magnitudes?
Rethinking Sequence Compression with Test-time Regression
Show answer and example
Answer: Unit keys group by direction; targets retain token strength. A static projection from the same normalized keys could be fit trivially and lose input-dependent compression. Per-forward coefficients do not update persistent parameters.
Illustrative example: Same-direction tokens of norm one and three have the same key but different targets; normalizing both would erase that distinction.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Implementation details
Why solve a T×T token-space system, and why not detach?
Rethinking Sequence Compression with Test-time Regression
Show answer and example
Answer: Scale K by D inverse square root, solve I_T plus the scaled K Gram matrix by Cholesky, then map to d dimensions. The outer loss needs solve gradients for projection and regularization. Complexity retains a cubic T term.
Illustrative example: Token-space is attractive when T≪d but is not always only O(T²d).
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Method comparison
How does TTR differ from fixed mean pooling?
Rethinking Sequence Compression with Test-time Regression
Show answer and example
Answer: It formulates and exactly solves a group-dependent ridge regression. Differentiating through the solve trains outer parameters; this is a different compression objective, not just new weights.
Illustrative example: Groups with equal means but different directional correlations can yield different coefficients.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Basics
Does test-time compression retrain model parameters at inference?
Rethinking Sequence Compression with Test-time Regression
Show answer and example
Answer: It solves coefficients anew for each token group. Outer loss learns target projection and regularizer; closed-form per-input fitting does not update those parameters at inference.
Illustrative example: Two inputs share a and ρ but solve different h.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Core mechanism
Why not fit token reverse KL on old rollout states alone?
Reverse-KL Policy Distillation: Statistical Guarantees and Limits
Show answer and example
Answer: Risk is under the candidate's visitation distribution. The method uses clipped prefix importance ratios and cumulative KL increments for bounded candidate comparison, then updates whole-policy weights. Exclude current action and weight both sides of the difference identically.
Illustrative example: A failure branch often visited by the old policy can be rare for the candidate and should not dominate.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Implementation details
Why exclude the current action and apply the same prefix ratio to both terms?
Reverse-KL Policy Distillation: Statistical Guarantees and Limits
Show answer and example
Answer: r_h multiplies actions only at u<h and weights the difference of candidate and rollout cumulative KL increments. Including the current action or weighting one side changes the bounded objective.
Illustrative example: At state three, the arrival ratio uses only actions one and two.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Method comparison
How does full-feedback teacher information differ from a sampled answer?
Reverse-KL Policy Distillation: Statistical Guarantees and Limits
Show answer and example
Answer: Complete action probabilities enable reverse KL and candidate comparisons at visited states. A single sample does not give them. Teacher support must include candidate-positive actions or KL can be infinite.
Illustrative example: Seeing A sampled reveals neither B nor C probabilities.
Sections in the Chinese source: 核心公式, 本地复现. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Basics
Why care which states the student visits?
Reverse-KL Policy Distillation: Statistical Guarantees and Limits
Show answer and example
Answer: Accumulate teacher/student KL under candidate-specific visitation. Equal weighting of all historical states can misrepresent actual risk.
Illustrative example: A branch nearly never visited by a candidate should not count equally merely because it occurs in old rollouts.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Core mechanism
Why use a count-weighted proxy for unread KV in OpenAttention?
OpenAttention: One Percent Is Enough with Tail Proxies
Show answer and example
Answer: Top-k discards all unread softmax mass. Query-relevant geometry groups tokens, with more exact reads where estimated error is high. Residual mean keys/values enter joint normalization with log counts. This preserves approximate mass, not exact attention.
Illustrative example: A mean proxy for 100 similar tail tokens needs count correction or it represents just one entry.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Implementation details
Why recompute cluster residuals after selecting exact tokens?
OpenAttention: One Percent Is Enough with Tail Proxies
Show answer and example
Answer: Otherwise exact rows are counted twice. Remove them, recompute means and counts, add log count to logits, and omit empty clusters. Keep documented deterministic boundaries rather than heuristic fixes.
Illustrative example: Selecting three of ten leaves a seven-item proxy.
Sections in the Chinese source: 核心公式, 本地复现. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Method comparison
What does a residual proxy retain beyond ordinary top-k?
OpenAttention: One Percent Is Enough with Tail Proxies
Show answer and example
Answer: The residual mean and count approximate omitted contributions and share softmax with exact tokens. The mean is not exact recovery.
Illustrative example: A proxy for 100 remaining tokens should not participate with single-token mass.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Basics
Why not read only a few important KV entries?
OpenAttention: One Percent Is Enough with Tail Proxies
Show answer and example
Answer: A low read rate can leave substantial total mass in the tail. Counted residual-cluster averages supplement exact reads instead of treating the tail as zero.
Illustrative example: Many individually weak tokens can jointly affect the output.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Core mechanism
Why does LazyFlash save backward work rather than visibility?
LazyFlash: Forward-Dense, Backward-Sparse Attention for Accelerated Long-Context Fine-Tuning
Show answer and example
Answer: Forward stays dense. Output-gradient block norms set backward retention probabilities, and inverse-probability weighting estimates dense gradients. A tiled backward kernel must skip actual work; masking afterward does not. A single random draw need not equal dense gradients.
Illustrative example: A low-contribution query block can be skipped in backward even though forward reads all evidence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Implementation details
Why isn't masking gradients after dense backward a LazyFlash kernel?
LazyFlash: Forward-Dense, Backward-Sparse Attention for Accelerated Long-Context Fine-Tuning
Show answer and example
Answer: Compute and memory traffic have already occurred. Skip tiles inside FlashAttention backward. No local operator or GPU receipt is claimed.
Illustrative example: Discarding half the results after computing all blocks saves no backward execution.
Sections in the Chinese source: 背景与主要改动, 本地复现. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Method comparison
Is importance backward equivalent to fixed top-k gradient truncation?
LazyFlash: Forward-Dense, Backward-Sparse Attention for Accelerated Long-Context Fine-Tuning
Show answer and example
Answer: No. Inverse-probability reweighting targets an unbiased dense-gradient estimate in expectation; fixed truncation generally does not. Individual draws still differ.
Illustrative example: A block kept with probability 0.5 is upweighted when sampled, not permanently dropped.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Basics
Why preserve dense forward but compress backward?
LazyFlash: Forward-Dense, Backward-Sparse Attention for Accelerated Long-Context Fine-Tuning
Show answer and example
Answer: Forward determines visible context. Uneven gradient contributions let training sample/reweight backward blocks without making inference forward sparse.
Illustrative example: Read the whole document, then sample only some backward blocks.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Structured Decisions / Decisions on Demand: Selective Reasoning, Transfer, and Feedback
Core mechanism
Why assess Jev-DLC reliability before updates and apply feedback only later?
Jev-DLC: Decision-Level Continual Learning from Typed Feedback
Show answer and example
Answer: Reliability measures prediction before feedback, not after learning the answer. The conservative state tracks repeated semantic experience, the responsive state tracks current correction, and reliability mixes them. Feedback must bind a committed decision; no forgery/replay or reading current gold before prediction.
Illustrative example: Example: A user undo can update both channels for the next request, not retroactively turn this choice into a success.
Sections in the Chinese source: 背景与主要改动, 双通道、PA-I与可靠性. The example illustrates the mechanism.
Structured Decisions / Decisions on Demand: Selective Reasoning, Transfer, and Feedback
Implementation details
Why reject feedback on forged, replayed or stale Jev-DLC Decisions?
Jev-DLC: Decision-Level Continual Learning from Typed Feedback
Show answer and example
Answer: Feedback must refer to a committed decision from this instance; otherwise arbitrary targets or duplicated experience can alter state. Release the current public cache after update. Adaptation never overrides hard safety masks.
Illustrative example: Example: Sending the same undo event twice must not count as two corrections.
Sections in the Chinese source: 背景与主要改动, 双通道、PA-I与可靠性. The example illustrates the mechanism.
Structured Decisions / Decisions on Demand: Selective Reasoning, Transfer, and Feedback
Method comparison
Why separate conservative and responsive Jev-DLC channels?
Jev-DLC: Decision-Level Continual Learning from Typed Feedback
Show answer and example
Answer: The conservative one builds a stable prior from repeated behavior; the responsive one tracks recent corrections. Reliability mixing balances long-term patterns and change; one state struggles to express both.
Illustrative example: Example: A long-standing preference for A and temporary switch to B can be recorded separately.
Sections in the Chinese source: 背景与主要改动, 双通道、PA-I与可靠性. The example illustrates the mechanism.
Structured Decisions / Decisions on Demand: Selective Reasoning, Transfer, and Feedback
Basics
Does Jev-DLC continual adaptation need to retain complete histories?
Jev-DLC: Decision-Level Continual Learning from Typed Feedback
Show answer and example
Answer: No. A frozen scorer has two compact states: semantic counts/direction for the conservative channel and a current correction direction for the responsive channel. Feedback enters state, not an ever-growing conversation prefix.
Illustrative example: Example: Repeated routing corrections update counts and directions rather than append every old ticket.
Sections in the Chinese source: 背景与主要改动, 双通道、PA-I与可靠性. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Core mechanism
Why use a differentiable online future encoding in the inverse-dynamics head instead of EMA?
Inside-JEPA: From Prediction to Relational Grounding in Latent World Models
Show answer and example
Answer: The inverse objective must send action-explanation gradients into relationship representations on the future side. JEPA prediction can still use a stable EMA target. Entity-only and shuffled-action controls test whether relations truly supply action information.
Illustrative example: Predict the action from changes in cup–box relations; the future relation encoder needs gradients to learn such changes.
Sections in the Chinese source: 背景与主要改动, 核心公式与五臂. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Implementation details
Why is the four-entity K=8 state 256-dimensional, not 224 after removing diagonals?
Inside-JEPA: From Prediction to Relational Grounding in Latent World Models
Show answer and example
Answer: Entity latents occupy 4×32 and ordered pair slots occupy 4×4×8. Padding and diagonals stay present but zero. JEPA uses detached EMA future targets; inverse dynamics needs differentiable online future state. Mixing them cuts a defining gradient.
Illustrative example: Zeroing diagonal relationships does not delete their 32 slots or shrink the predictor interface.
Sections in the Chinese source: 核心公式与五臂, 本地复现与操作. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Method comparison
What do INV-S, INV-R, and shuffled-label controls isolate?
Inside-JEPA: From Prediction to Relational Grounding in Latent World Models
Show answer and example
Answer: INV-S reads relations only for inverse dynamics; INV-R reads entity latents only. Shuffling changes only inverse labels, while prediction still reads true actions. These controls isolate supervision sources rather than treating any auxiliary gain as relation grounding.
Illustrative example: Similar gains with random action labels suggest regularization rather than learned action–relation correspondence.
Sections in the Chinese source: 背景与主要改动, 核心公式与五臂. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Basics
Why does Inside-JEPA want latent states to encode relationships?
Inside-JEPA: From Prediction to Relational Grounding in Latent World Models
Show answer and example
Answer: Latent forecasts need not expose contact or relative position. Inside-JEPA models entities and relations and studies how action supervision shapes them. A relation channel or low prediction loss does not establish learned relational semantics; controls are needed.
Illustrative example: Two unchanged-looking objects can transition from separated to touching, altering the right action despite unchanged categories.
Sections in the Chinese source: 背景与主要改动, 核心公式与五臂. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why does FineSID need more than SID embeddings when transferring relevance pretraining to CTR?
Progressive Learning from Content to Relevance to Personalization for CTR Prediction
Show answer and example
Answer: Relevance pretraining teaches ordered content fusion and HiFormer interactions across embeddings, dense layers, and normalization state. Copying only the vocabulary loses much of that learning. Change the CTR head and handle optimizer state explicitly. Full experiments were excluded for cost; random-token fixtures are not substitutes.
Illustrative example: Copying codes without the layers that match query uses to image details transfers a dictionary but not how to read it.
Sections in the Chinese source: 背景与主要改动, 三阶段结构, 成本判定：不执行复现. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
Why explicitly handle optimizer and normalization state at FineSID stage transitions?
Progressive Learning from Content to Relevance to Personalization for CTR Prediction
Show answer and example
Answer: Full transfer includes interaction layers and normalization statistics beyond SID embeddings. Keeping or resetting optimizer state changes training dynamics and must follow the stage contract, as must conversion to the CTR head. These requirements are documented; no local training validates performance.
Illustrative example: Copying weights without their normalization can change scale abruptly, and old task momentum should not silently enter a new CTR head.
Sections in the Chinese source: 三阶段结构, 成本判定：不执行复现. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
How does full FineSID predictor transfer differ from freezing a content encoder and adding a CTR head?
Progressive Learning from Content to Relevance to Personalization for CTR Prediction
Show answer and example
Answer: A frozen encoder mainly transfers content features; FineSID also retains cross-token fusion, interaction networks, and state learned on relevance, then adapts to personalized CTR. This costs more and requires matched interfaces. The repository completed cost review, not a reduced-model performance run.
Illustrative example: Transfer the matching network that relates query conditions to item details, not just a content vocabulary.
Sections in the Chinese source: 背景与主要改动, 三阶段结构, 成本判定：不执行复现. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why keep fine-grained visual and text tokens rather than a single product code in FineSID?
Progressive Learning from Content to Relevance to Personalization for CTR Prediction
Show answer and example
Answer: A single code can discard local texture, regions, and text details that drive relevance and clicks. Discrete local tokens support richer interactions through relevance pretraining but cost more in training and serving. More granularity does not guarantee better recommendations.
Illustrative example: Two coats differ mainly in cuffs or pockets; a single summary code may lose the deciding detail.
Sections in the Chinese source: 背景与主要改动, 三阶段结构, 成本判定：不执行复现. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Core mechanism
Why skip correction for successes and select only some tokens from failures?
Edit, Don't Imitate: Minimal-Edit On-Policy Distillation for Multimodal Reasoning
Show answer and example
Answer: Preserve successful student behavior with task reward. Compare teacher and old student only for errors, correcting less teacher-supported tokens. Full group advantage precedes split. Reference solution and full reply are teacher training inputs only.
Illustrative example: A misread geometry label can receive focused suppression without reworking a successful alternative response.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Implementation details
Compute ME-OPD group advantages before or after splitting successes and failures?
Edit, Don't Imitate: Minimal-Edit On-Policy Distillation for Multimodal Reasoning
Show answer and example
Answer: Use all K rewards before splitting. A success-only group has constant reward. Use frozen old student for strict-greater mask and divide by selected correction tokens across the minibatch.
Illustrative example: Teacher and old student ties are not selected; do not remake mask from updated scores.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Method comparison
How do ME-OPD and Projected Distillation produce corrections differently?
Edit, Don't Imitate: Minimal-Edit On-Policy Distillation for Multimodal Reasoning
Show answer and example
Answer: ME-OPD teacher-forces and suppresses chosen original error tokens. Projected Distillation generates a verified revision and compares edited spans via DPO. Scoring existing tokens and producing paired text differ.
Illustrative example: Only the latter needs a newly written answer aligned to the original.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Basics
Why skip teacher correction for correct ME-OPD replies?
Edit, Don't Imitate: Minimal-Edit On-Policy Distillation for Multimodal Reasoning
Show answer and example
Answer: Successful traces retain task reward without stylistic interference. Failures alone select original tokens the teacher supports less. The teacher scores existing responses instead of writing new ones.
Illustrative example: Do not rewrite a valid alternative solution just because its style differs.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
Why project one gradient relative to a chosen protected gradient?
CORD: Conflict-Aware Optimization for Joint RL and Distillation in Agents
Show answer and example
Answer: Minimally change the conflicting other vector to preserve the selected objective. Swapping protection changes updates. Conflict-only and always differ; project after full-batch accumulation. This is pre-optimizer first-order geometry, not guaranteed finite AdamW descent.
Illustrative example: Task reward and imitation may push a parameter oppositely; changing which is protected changes what survives.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
Why aggregate a full CORD optimizer batch before projecting once?
CORD: Conflict-Aware Optimization for Joint RL and Distillation in Agents
Show answer and example
Answer: Projection is nonlinear, so per-microbatch projection and summation usually differ. After fixed aggregation apply objective weight λ first. Skip projection if protected gradient is zero; λ=0 returns first objective direction.
Illustrative example: Conflicts in two microbatches can cancel in the aggregate; early projection changes the result.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
How do CORD conflict-only and always-project variants differ?
CORD: Conflict-Aware Optimization for Joint RL and Distillation in Agents
Show answer and example
Answer: Conflict gating keeps helpful aligned components and projects only for negative dot product. Always projecting removes even helpful parallel components. The resulting direction is different; no finite adaptive-optimizer descent guarantee.
Illustrative example: Aligned gradients add directly under conflict gating; always-project still removes a parallel component.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
Why does CORD select a protected objective first?
CORD: Conflict-Aware Optimization for Joint RL and Distillation in Agents
Show answer and example
Answer: Adding conflicting task and teacher gradients can cancel them. The protected direction states whose first-order progress must not be harmed; modify the other minimally. It is a tradeoff.
Illustrative example: Protecting task reward versus teacher imitation defines different projections.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Core mechanism
Why allow bidirectionality within AW-0 experts but enforce code→video→action across them?
Show answer and example
Answer: Bidirectionality organizes representations inside a modality; block causality preserves conditioning from plan to scene prediction to control. Code must not see later videos or actions, while action can use earlier code and predicted scenes. Full mutual visibility changes inference order and available information.
Illustrative example: Generate a cup-grasp program, predict the scene, then output controls; do not give the planner future ground-truth actions.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Implementation details
Can flow targets and masked code diffusion use one noise and supervision mask?
Show answer and example
Answer: No. Video and action use continuous noising and velocity regression. Code uses an independent Bernoulli mask and loss only on masked program tokens. Action noise and code masking are not determined by video noise. Mixed video sampling also needs separate clipped and normalized importance weights.
Illustrative example: Low video noise does not imply lightly masked code; one shared mask changes the paper's independent sampling.
Sections in the Chinese source: 本地收录与复现边界. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Method comparison
How do separate expert projections, normalization, and FFNs differ from fully shared transforms?
Show answer and example
Answer: Separate transforms serve code, video, and action while joint attention exchanges information at prescribed boundaries. They add parameters and interfaces, not guaranteed improvement. The paper combines bidirectional intra-expert and block-causal cross-expert attention.
Illustrative example: Discrete code and continuous actions can use separate FFNs and still interact, without being isolated or freely bidirectional.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Basics
Why combine composable programs, future vision, and robot action in one AW-0 model?
Show answer and example
Answer: Programs provide reusable steps, video anticipates scenes, and continuous actions control the robot. Three experts jointly link plan to physical consequences; generated code plans by default rather than being executed. Local work reviews evidence only.
Illustrative example: Generate “grasp then place,” forecast the scene, and produce controls without running arbitrary Python.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Core mechanism
Why distill on the quantized student's prefixes with per-candidate KL mixing?
Right On Q: Recovering Reasoning under Quantization via Self-Generated Trajectories
Show answer and example
Answer: Quantization changes visited states, so score the student's own prefixes. Teacher/student probability ratios mix forward and reverse KL per candidate to treat over- and underestimation differently. Detach mixture weights and retain a residual mass bucket.
Illustrative example: A correct number underweighted and a connective overweighted at one position need different coefficients, not ideal teacher prefixes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Implementation details
Can OnQ renormalize only the union of teacher and student top-20?
Right On Q: Recovering Reasoning under Quantization via Self-Generated Trajectories
Show answer and example
Answer: No. Keep a remainder bucket with student residual gradients and teacher smoothing. Truncation would erase tail differences. OPD updates latent weights and scales, with fixed zero point.
Illustrative example: If the union covers 90%, the remaining 10% stays a class.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Method comparison
How does candidate Dynamic KL differ from one fixed mixture per response?
Right On Q: Recovering Reasoning under Quantization via Self-Generated Trajectories
Show answer and example
Answer: At a prefix, some candidates are underweighted and others overweighted. α_v adapts to p_v/q_v per candidate and is detached to avoid optimizing the weight itself.
Illustrative example: Increasing support for one candidate and suppressing another need not share a mixture.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Basics
Why let the quantized student generate training prefixes?
Right On Q: Recovering Reasoning under Quantization via Self-Generated Trajectories
Show answer and example
Answer: Quantization changes visited reasoning states. Query the frozen teacher on the student's actual states rather than fixed full-precision answers.
Illustrative example: If the student takes a wrong turn, the teacher still scores the same prefix.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Core mechanism
Why reserve routing anchors in a normal MoE KV compression budget?
MoEZip: Routing-Aware KV Cache Compression for Sparse Mixture-of-Experts LLMs
Show answer and example
Answer: KV deletions can change attention and subsequent routing. Propagate Fisher sensitivity from observed instruction positions through attention, reserve anchors, then fill with the base compressor. Sensitivity is a proxy, not a route-flip guarantee or permission to read gold.
Illustrative example: An older low-attention token can still affect expert choice and merit retention.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Implementation details
Can the base compressor select anchors again when filling budget?
MoEZip: Routing-Aware KV Cache Compression for Sparse Mixture-of-Experts LLMs
Show answer and example
Answer: No. Exclude already selected anchors and fill with distinct positions. Gather real KV in original temporal order. Use prefill instruction only, never future gold diagnostics.
Illustrative example: With 64 slots and 16 anchors, add 48 distinct non-anchor positions.
Sections in the Chinese source: 核心公式, 本地复现. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Method comparison
Is Fisher trace an exact top-k router flip criterion?
MoEZip: Routing-Aware KV Cache Compression for Sparse Mixture-of-Experts LLMs
Show answer and example
Answer: No. It averages distribution curvature over directions and omits the router-to-hidden Jacobian. High trace does not imply a flip.
Illustrative example: Equal trace at two positions can yield different routing changes under different perturbation directions.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Basics
Why connect KV error to MoE routing?
MoEZip: Routing-Aware KV Cache Compression for Sparse Mixture-of-Experts LLMs
Show answer and example
Answer: Eviction perturbs attention outputs seen by the router. Attention quality alone can miss route-sensitive positions, so estimate anchors from observed instruction gates and attention.
Illustrative example: A nonmaximal KV can move a token across an expert boundary.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Structured Decisions / Candidate Invariance: Order, Competitors, and Calibration
Core mechanism
Why can exact-set recovery fail even when every correct option outranks distractors?
Show answer and example
Answer: One correct choice can absorb most listwise mass, leaving another below threshold or the uniform baseline. Correct ranking differs from exact-set recovery, and shares are not independent probabilities. Fit deployment thresholds on validation only.
Illustrative example: Example: Two correct options rank first and second, but uneven shares cause thresholding to keep only one. A top-one hit misses the incomplete set.
Sections in the Chinese source: 背景与主要改动, 方法与本地协议. The example illustrates the mechanism.
Structured Decisions / Candidate Invariance: Order, Competitors, and Calibration
Implementation details
May the threshold be reselected from the test exact-set curve?
Show answer and example
Answer: No. Deployment selects thresholds on a validation grid and freezes them for test. A same-data threshold curve is retrospective. The predictor does not see the correct set.
Illustrative example: Example: Picking the best point after seeing test is a hindsight upper bound, not unseen-data performance.
Sections in the Chinese source: 背景与主要改动, 方法与本地协议. The example illustrates the mechanism.
Structured Decisions / Candidate Invariance: Order, Competitors, and Calibration
Method comparison
Why calibrate listwise shares differently from independent binary probabilities?
Show answer and example
Answer: Listwise shares depend on other options and sum to one; multiple binary probabilities may be simultaneously high. One threshold or calibration meaning confuses competition with correctness.
Illustrative example: Example: Adding another correct option can reduce one item's listwise share without reducing its binary probability.
Sections in the Chinese source: 背景与主要改动, 方法与本地协议. The example illustrates the mechanism.
Structured Decisions / Candidate Invariance: Order, Competitors, and Calibration
Basics
Does a multicorrect task require one answer or the entire correct set?
Show answer and example
Answer: Recover all correct candidates; there may be more than one. One correct choice can miss others; choosing everything can include distractors. Interpret ranking, thresholding and exact-set metrics using a set rather than a winner.
Illustrative example: Example: Correct {A,B} is not recovered by {A} or {A,B,C}.
Sections in the Chinese source: 背景与主要改动, 方法与本地协议. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Core mechanism
Why is zeroing a DetectorScope concept exact in logit space but not equivalent to editing text?
Building Auditable AI Text Detectors with Sparse Autoencoders
Show answer and example
Answer: Sparse pooled features feed a linear head, so each weight times activation is an exact contribution. Text edits can change several features and contexts. Automatic descriptions are imperfect; local head excludes the full SAE/explanation pipeline.
Illustrative example: Zero one feature to verify its score change; deleting a word can alter several concepts.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Implementation details
Why not apply an L1 subgradient and soft thresholding together?
Building Auditable AI Text Detectors with Sparse Autoencoders
Show answer and example
Answer: Apply BCE SGD with momentum, then ηλ soft thresholding, without a second L1 gradient or bias penalty. Fit uncentered RMS on training only.
Illustrative example: Penalizing twice changes effective sparsity beyond λ.
Sections in the Chinese source: 核心公式, 数据协议与结果. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Method comparison
Is description validation equivalent to human confirmation of concept semantics?
Building Auditable AI Text Detectors with Sparse Autoencoders
Show answer and example
Answer: Spearman correlation compares observed and predicted activation ranks as a filter, not manual validation or safety. Constant sequences do not support the correlation.
Illustrative example: Correlation on a small sample does not prove meanings are correct for every text.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Basics
Why make detection depend on inspectable sparse features?
Building Auditable AI Text Detectors with Sparse Autoencoders
Show answer and example
Answer: Pool SAE features, filter unreliable concept descriptions, and fit a sparse linear head so logits decompose into actual computed contributions rather than post-hoc reasons.
Illustrative example: Verify a contribution by zeroing its feature rather than trusting generated explanations.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Core mechanism
Why supervise only teacher-crop B patches that fall inside student crop A in stage two?
ALPS: Visual State Encoders for World Models
Show answer and example
Answer: The student sees only A, so exact targets outside A would supervise invisible content. Map geometry first, then predict frozen teacher targets at corresponding positions; appearance augmentation promotes robustness. Shared initialization does not mean the teacher updates with the student.
Illustrative example: If the student sees the left half, supervise only the overlap, not small text on the unseen right.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Implementation details
Why average the cross-crop loss over valid patches per image before averaging the batch?
ALPS: Visual State Encoders for World Models
Show answer and example
Answer: Overlap sizes differ; per-image means prevent large overlaps dominating. A B patch is valid if its center falls in A's half-open bounds. Empty overlap safely contributes zero, while the sampler separately enforces a minimum valid count.
Illustrative example: With 100 matching patches in one image and ten in another, a flat token mean gives tenfold weight to the first; image means do not.
Sections in the Chinese source: 核心公式, 本地复现, 验证与结果边界. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Method comparison
How does the fixed stage-one teacher differ from an EMA teacher?
ALPS: Visual State Encoders for World Models
Show answer and example
Answer: Stage one initializes the student and a teacher frozen through stage two. EMA would move targets with student weights, which is a different method. Training drops the cross-view predictor afterward and deploys the student encoder and language interface.
Illustrative example: Student augmentation training may change the encoder while fixed clean-crop teacher targets remain unchanged.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Basics
Why does ALPS want one visual representation for language, state prediction, and image inspection?
ALPS: Visual State Encoders for World Models
Show answer and example
Answer: Planning needs predictability, instruction tasks need language alignment, and inspection needs readable image recovery. Shared patch tokens support those interfaces, but each must be tested. Pixel reconstruction alone does not establish planning, nor language alignment complete physical state.
Illustrative example: A robot needs to understand “move the cup right,” predict its location, and inspect the future image.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Core mechanism
Why use hard gate G to cut off objective credit instead of letting high objective values offset failed hard constraints?
PLE: Physical-Engineering's Last Exam
Show answer and example
Answer: Engineering artifacts may optimize a target while violating requirements. All hard constraints must pass for the gated reward. Constraint ratios retain component information, and missing tasks score zero in the full denominator. Local aggregation consumes external grades, not independent physical or regulatory judgments.
Illustrative example: A lightweight design that fails a load-bearing constraint cannot compensate with weight savings. Missing artifacts cannot be dropped from the average.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Implementation details
Why retain the complete task denominator and missing items in PLE aggregation?
PLE: Physical-Engineering's Last Exam
Show answer and example
Answer: Average each run over all tasks with missing scores at zero, then average three run means. Conditioning on successful submissions inflates results. Local scoring accepts external grades and artifact bytes, not independent certification of physical or regulatory correctness.
Illustrative example: If only 10 of 85 tasks are complete, their average is not the full benchmark mean.
Sections in the Chinese source: 核心公式, 离线验收接口. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Method comparison
How does PLE's hard-gated constraint score differ from scoring continuously by objective value alone?
PLE: Physical-Engineering's Last Exam
Show answer and example
Answer: When any hard constraint fails, G=0 and objective optimization earns nothing; higher objective values cannot offset the violation. Expert rubrics and special site/MEP graders have separate rules, so one formula does not cover every task.
Illustrative example: High efficiency cannot earn the 0.6GO′ term after any hard-constraint failure.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Basics
Why does PLE require auditable engineering artifacts rather than only written answers?
PLE: Physical-Engineering's Last Exam
Show answer and example
Answer: Long engineering tasks have multimodal inputs, constraints, and deliverable files. Experts or independent graders check whether actual outputs meet requirements. Fluent descriptions do not establish completed designs.
Illustrative example: A written explanation does not prove the corresponding engineering file exists or passes constraints.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
Why let the student explore before learning the expert's full successful trajectory for the same hidden task?
Asymmetric On-Policy Distillation Learns In-Context Exploration
Show answer and example
Answer: An expert knows the hidden goal, so a direct success omits the context of continuing after mistakes. Condition expert-action prediction on the student's actual history, teaching task inference from observations and rewards. The hidden goal stays unavailable during deployment, and student exploration actions are not automatically correct labels.
Illustrative example: After the student searches two wrong rooms, the expert's success on the same task can teach how to narrow the search from that history rather than merely memorize a direct opening.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
How does the local single-step three-door diagnostic preserve complete expert trajectories, and what are its limits?
Asymmetric On-Policy Distillation Learns In-Context Exploration
Show answer and example
Answer: Each door choice is a complete one-step episode, so expert length one reflects the environment rather than general simplification of a multistep method. A real small Transformer is supervised only on the terminal expert query, not student-history actions. Deployment starts with empty history and no expert; there is no general POMDP or robotics comparison.
Illustrative example: An incorrect student attempt stays in context, but loss does not train it to repeat the error. The correct expert action is stored separately as the training target.
Sections in the Chinese source: 核心公式, 本地复现, 保留的定义机制与缩小边界. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
Why shouldn't cross-episode BAMDP conditioning and generic POMDP takeover be reduced to single-step DAgger?
Asymmetric On-Policy Distillation Learns In-Context Exploration
Show answer and example
Answer: A fixed-latent BAMDP accumulates task knowledge across episodes and pairs it with an expert episode on the same task. A generic POMDP hands over at an intermediate student state. Complete expert traces and context are defining; a single current-action label discards them. Deployment cannot keep invoking a latent-aware expert.
Illustrative example: If the hidden door stays the same across three attempts, first-round failure informs the next. If the door changes each round, the same cross-episode history no longer conveys that knowledge.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why might imitating expert successes be insufficient in a hidden-task environment?
Asymmetric On-Policy Distillation Learns In-Context Exploration
Show answer and example
Answer: The student tries imperfect actions and encounters information and states absent from smooth expert paths. ASTEROID combines student exploration with expert success to cover actual uncertainty. Exploration uses visible interaction, and expert training information remains isolated rather than exposing hidden tasks or solutions during evaluation.
Illustrative example: The expert takes the right path immediately, while the student tries the wrong door first. Learning only the expert's opening does not show the student how to use observed failure information.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why freeze centers and use a load bound accounting for the heaviest item?
SPAR: Stepwise Personalized Autoregressive Recommendation with Prefix-Balanced Semantic IDs
Show answer and example
Answer: Fixing centers and parents limits semantic drift. Indivisible weights can make exact averages infeasible; move at most once per item and choose low-cost feasible destinations. Local scope excludes retrieval and preference alignment.
Illustrative example: Total load 12 across three branches cannot be exactly four each if one item weighs six.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Does local SPAR implement PATH retrieval and full SFT+DPO?
SPAR: Stepwise Personalized Autoregressive Recommendation with Prefix-Balanced Semantic IDs
Show answer and example
Answer: No. It implements post-freeze RQ-PIB only. Skip zero-load groups, move items at most once, and keep parents and centers fixed.
Illustrative example: Local branch balancing does not train prefix personalization.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
How do feasible RQ-PIB moves differ from global equal buckets?
SPAR: Stepwise Personalized Autoregressive Recommendation with Prefix-Balanced Semantic IDs
Show answer and example
Answer: Choose a currently feasible low-distortion move under load limits and indivisible-item weights. It guarantees neither exact equality nor global optimum.
Illustrative example: A very popular item cannot be split across branches merely to equalize loads.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Why repair SPAR load within each parent SID prefix?
SPAR: Stepwise Personalized Autoregressive Recommendation with Prefix-Balanced Semantic IDs
Show answer and example
Answer: Compare sibling load by interaction weight, not item count, while freezing centers and preserving hierarchical semantics through low-distortion moves.
Illustrative example: Two branches with ten items each may have very different traffic.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why does ECHO distill the full teacher distribution and also penalize misses near the selection boundary?
ECHO: Efficient Cascade with Homogeneous Operators for Distillation-Guided 100K-Sequence Modeling
Show answer and example
Answer: Soft targets communicate relative history relevance but can leave crucial records outside a limited budget. Boundary pairs compare a better missed record against a weaker selected one, using detached teacher-score differences to drive swaps. The component retains addresses, and the local run verifies loss and gradients, not 100K-history performance.
Illustrative example: With budget two, the student misses a highly valued record and keeps a weak one. KD conveys overall order; the pair directly promotes the swap.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why normalize ECHO by valid pair weights across the batch and detach teacher values and weights?
ECHO: Efficient Cascade with Homogeneous Operators for Distillation-Guided 100K-Sequence Modeling
Show answer and example
Answer: Averaging by row can inflate requests with few pairs. Batch normalization preserves defined pair contributions. Freeze teacher and mining weights, update matching student scores only, and exclude padding from probabilities, selection, and gradients. A finite scalar alone is insufficient verification.
Illustrative example: With ten pairs on one row and one on another, per-row means can give the lone pair tenfold relative weight; a shared denominator avoids that.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How do ECHO boundary pairs and full-distribution KD supervise different errors?
ECHO: Efficient Cascade with Homogeneous Operators for Distillation-Guided 100K-Sequence Modeling
Show answer and example
Answer: KD transfers overall teacher preference; pairing missed strong records with selected weak ones focuses on swaps that change Top-K. Teacher quality gaps weight pairs. The discrete selection remains nondifferentiable; selected student logits receive gradients.
Illustrative example: Positions ten and eleven can have similar global probabilities yet determine whether a vital record is kept. Boundary loss targets that pair.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why retain history record addresses rather than only latent interest vectors in ECHO?
ECHO: Efficient Cascade with Homogeneous Operators for Distillation-Guided 100K-Sequence Modeling
Show answer and example
Answer: Latents are compact but cannot recover concrete candidate-specific events. Addressable records enable targeted small-subset retrieval and user-side reuse. This does not imply scanning all history per candidate; serving must follow the exported index and parameter path, beyond the local loss component.
Illustrative example: A camera query can retrieve a recent macro-photography view and running shoes can retrieve exercise evidence, neither recoverable from a vague outdoors vector.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Core mechanism
Why keep the raw prediction after subtracting a user or item baseline?
Baseline and Incremental: User, Item, and Request Debiasing in Instagram Feed Ranking
Show answer and example
Answer: The increment measures how far above normal a candidate is, while the raw score preserves absolute business magnitude. The method separately weights a nonlinear increment: compare probabilities in log-odds space and use same-unit differences for regressions. This is a scoring configuration, not causal uplift.
Illustrative example: A small lift over a low baseline can outrank a more valuable candidate on relative change alone; the raw score keeps the high baseline's value visible.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Implementation details
Why not implement Baseline-Incremental's demotion gate as the negative of its promotion gate?
Baseline and Incremental: User, Item, and Request Debiasing in Instagram Feed Ranking
Show answer and example
Answer: When the promotion amplitude is prediction p, it naturally vanishes near zero. Negating it would barely penalize the low-p candidates that most need demotion. Preserve the constant and gap amplitudes and piecewise limits from the source, explicitly clip probabilities before taking logits, and verify the gates separately.
Illustrative example: A candidate with p near zero receives almost no penalty from −p, while constant demotion can still suppress it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Method comparison
Why use log-odds differences for probabilities but original-unit differences for regression?
Baseline and Incremental: User, Item, and Request Debiasing in Instagram Feed Ranking
Show answer and example
Answer: An equal absolute probability difference has a different meaning near zero or one; logit differences express change on the odds scale. Regression targets already have natural units. These distinct scoring responses are intentional and do not confer a causal interpretation.
Illustrative example: Click-through rates of 1% to 2% and 40% to 41% both gain one point but change odds differently; extra watch time can be compared directly in seconds.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Basics
What reference points do the user, item, and request baselines represent in Baseline-Incremental?
Baseline and Incremental: User, Item, and Request Debiasing in Instagram Feed Ranking
Show answer and example
Answer: The user baseline captures customary response, the item baseline broad attractiveness, and the request baseline the current candidate pool's level. Increment scores show how a candidate stands out; raw scores keep absolute value. This is not causal uplift, and a different reference changes the ranking's meaning.
Illustrative example: A high-activity user's video click probability may only slightly exceed their norm, whereas a moderate probability for a less active user is more exceptional. Retain both absolute values.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Core mechanism
Why restrict DPO to edits and SFT to student successes?
Projected Distillation: On Learning to Solve Hard Problems from Close, Correct Targets
Show answer and example
Answer: Minimal repair preserves generated structure and focuses preference credit on corrections. Repaired and independently correct responses have different provenance. Score both sides under their full causal contexts; shared parameters can still change.
Illustrative example: If only the last operator is wrong, verify its replacement and contrast the edit rather than imitate all teacher wording.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Implementation details
Can edited-span scoring drop the preceding text?
Projected Distillation: On Learning to Solve Hard Problems from Close, Correct Targets
Show answer and example
Answer: No. Score each edit under its own complete causal prefix, summing span logp without length averaging. One insertion/deletion side may be empty but pair needs an edit. Zero direct gradient elsewhere does not protect shared weights.
Illustrative example: Editing step three changes step four's context; don't borrow another response's prefix.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Method comparison
How does edited-span DPO differ from SFT on a complete repaired answer?
Projected Distillation: On Learning to Solve Hard Problems from Close, Correct Targets
Show answer and example
Answer: Span DPO contrasts only original and verified edit positions; unchanged text is not directly included. Full SFT is for student-generated successes, not projector repairs.
Illustrative example: An externally repaired reply is not an autonomous success from this round.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Basics
Why minimally repair a wrong student response before projected distillation?
Projected Distillation: On Learning to Solve Hard Problems from Close, Correct Targets
Show answer and example
Answer: Expert prose can be far from student distribution. Minimal edits retain current behavior while correcting what is needed, with independent verification. This is an approximation, not exact most-likely correct output.
Illustrative example: Correct one wrong symbol instead of replacing the whole derivation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Core mechanism
Why choose candidate tokens once at the final student layer?
On-Policy Transition Distillation
Show answer and example
Answer: Track the same predictions through depth. Independent top-k confounds identity with changes. Use full-vocabulary-normalized log-probability increments plus final OPD, with detach and destination-layer weights, not hidden MSE or exact transition KL.
Illustrative example: If shallow favors A and deep B, retain both identities to trace their rise/fall.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Implementation details
Which OPTD increment endpoints receive gradients and how is practical scale handled?
On-Policy Transition Distillation
Show answer and example
Answer: Both adjacent student readouts are differentiable. Teacher, candidate weights and correction coefficients are frozen. Appendix sums transitions while main text averages; neither is exact reverse transition KL.
Illustrative example: A sum of three transitions cannot silently become a mean at the same objective scale.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Method comparison
How does transition supervision differ from independent top-k at each layer?
On-Policy Transition Distillation
Show answer and example
Answer: Final student layer chooses fixed token identities for the entire path. Independent sets mix different objects when comparing increments.
Illustrative example: If shallow top is A and deep top B, subtracting their logp doesn't measure A's change.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Basics
Why doesn't inter-layer OPTD only align final predictions?
On-Policy Transition Distillation
Show answer and example
Answer: Final agreement can hide different depth-wise revisions. Supervise shallow-to-deep log-probability increments and retain final OPD. This is prediction paths, not hidden-vector matching.
Illustrative example: Two models can end on A after very different intermediate changes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Core mechanism
Why preserve REST across tokenizers rather than softmax common matches?
CSD: Common-Support Distillation for Cross-Tokenizer Knowledge Transfer
Show answer and example
Answer: Build common text events from verified decoded boundaries and chain probabilities. REST carries omitted mass; renormalization discards it and changes divergence. Don't align by position or IDs alone.
Illustrative example: Two tokens and one token may express one word, but unmatched continuation probability still exists.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Implementation details
Do CSD student gradients flow only through the final token of a chunk?
CSD: Common-Support Distillation for Cross-Tokenizer Knowledge Transfer
Show answer and example
Answer: No. Include prefix chain factors, final token and REST. Freeze teacher. Non-1:1 weights don't enter denominator count; fall back to CE for no KD/shared event and detach overall ratio.
Illustrative example: Ignoring the first of two pieces miscomputes its entire text event.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Method comparison
What does REST preserve that renormalizing common candidates removes?
CSD: Common-Support Distillation for Cross-Tokenizer Knowledge Transfer
Show answer and example
Answer: It retains probability of unmatched events instead of pretending their intersection is the whole distribution. Do not replace chain probability with a length mean.
Illustrative example: A small shared intersection may appear very similar after softmax while most mass was omitted.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Basics
Why align shared textual events rather than token IDs in CSD?
CSD: Common-Support Distillation for Cross-Tokenizer Knowledge Transfer
Show answer and example
Answer: Tokenizers differ in IDs, vocabulary and segmentation. Verify decoded character spans, then compare probability of common text events.
Illustrative example: One tokenizer can split a word in two; identical IDs wouldn't establish matching text.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Core mechanism
Why require disagreement and teacher support within student reach?
Ask to Distill: Learning to Synthesize Data for On-Policy Distillation
Show answer and example
Answer: Disagreement alone can choose unlearnable problems, compatibility alone mastered ones. Reward their product using raw teacher mass, freeze scorers, then distill from original student.
Illustrative example: A teacher refinement near two known approaches has useful gap; mass almost entirely off-support should score low.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Implementation details
Why not renormalize A2D compatibility on the candidate union?
Ask to Distill: Learning to Synthesize Data for On-Policy Distillation
Show answer and example
Answer: Use raw full-vocabulary teacher mass on student candidates; renormalization hides mass outside. Deduplicate across complete scoring batch, not one GRPO group; don't arbitrarily restandardize downstream advantages.
Illustrative example: If teacher mostly supports inaccessible tokens, local normalization overstates compatibility.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Method comparison
How does A2D compatibility differ from pure disagreement reward?
Ask to Distill: Learning to Synthesize Data for On-Policy Distillation
Show answer and example
Answer: A big gap can mean impossible. Compatibility measures teacher mass on student support. Product balances room for learning and reachability rather than reward distance alone.
Illustrative example: A teacher-only path can have high disagreement but poor teachability.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Basics
Why train an A2D question synthesizer before training student?
Ask to Distill: Learning to Synthesize Data for On-Policy Distillation
Show answer and example
Answer: Questions determine visited prefixes and usable guidance. Freeze scorer teacher/student, optimize question distribution, screen pool then distill from original student. Separate curriculum learning from student gains.
Illustrative example: Find learnable challenge problems without accumulating student updates while selecting them.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Core mechanism
Why use original block statistics and scale negative coefficients too?
Adaptive Pullback: Reweighting Supervision at Penalty Jumps in On-Policy Distillation
Show answer and example
Answer: Weighted statistics would recursively alter subsequent reference levels. Detect from original positive penalty means but attenuate all signed supervision in the later block. Preserve direction, allow zero weight, and do not interpret threshold as calibrated significance.
Illustrative example: Compare block three to block two's original level even if block two was already reduced.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Implementation details
Can the next AP-OPD boundary use previously attenuated statistics?
Adaptive Pullback: Reweighting Supervision at Penalty Jumps in On-Policy Distillation
Show answer and example
Answer: No. Always detect from original unweighted means/variance to avoid recursive distortion. No renormalization or hidden floor; previous mean zero may produce zero weight. Sample variance needs enough tokens.
Illustrative example: Detecting from an already reduced block can create artificial jumps.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Method comparison
How does AP-OPD attenuation differ from dropping outlier tokens?
Adaptive Pullback: Reweighting Supervision at Penalty Jumps in On-Policy Distillation
Show answer and example
Answer: Apply one factor to all signed coefficients in a block, preserving signs and including negatives. Dropping tokens is a different selection rule. The detector is not a calibrated significance test.
Illustrative example: Negative log ratios in the later block are also scaled.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Basics
Why does AP-OPD target penalty jumps rather than uniformly high penalties?
Adaptive Pullback: Reweighting Supervision at Penalty Jumps in On-Policy Distillation
Show answer and example
Answer: Persistent disagreement and sudden local deterioration differ. Compare adjacent original block statistics and attenuate the later block to limit spike dominance.
Illustrative example: A hard entire response differs from one getting abruptly worse after a particular step.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Structured Decisions / Typed Readout: From Generated Sentences to Candidate Distributions
Core mechanism
Why report both candidate probabilities and full-vocabulary label mass?
Multimodal Reranking Is a System One Decision
Show answer and example
Answer: Within-label normalization always sums to one, even when the model barely supports any legal label. Full-vocabulary mass exposes low support. Labels must be distinct single tokens; the local readout is not a separately trained System One model.
Illustrative example: Example: A can lead among letters even when their combined vocabulary mass is tiny. It may rank first without high absolute confidence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Structured Decisions / Typed Readout: From Generated Sentences to Candidate Distributions
Implementation details
Why must candidate labels be distinct single tokens under the current tokenizer?
Multimodal Reranking Is a System One Decision
Show answer and example
Answer: Readout uses one answer-position logit vector. Multi-token labels need conditional continuation scores, and another model's token IDs can name different tokens. Verify each label; do not silently use only the last piece.
Illustrative example: Example: If a new tokenizer splits a letter marker, reject it or define a different protocol rather than reuse old IDs.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Structured Decisions / Typed Readout: From Generated Sentences to Candidate Distributions
Method comparison
How does setwise readout differ from per-document yes/no scoring?
Multimodal Reranking Is a System One Decision
Show answer and example
Answer: Setwise options compete in one normalized context; binary scores judge relevance independently. Their denominators and interactions differ, so probabilities are not interchangeable. The paper's yes/no arm was not reproduced locally.
Illustrative example: Example: Two relevant documents can each have high yes probability, while two choices share total mass.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Structured Decisions / Typed Readout: From Generated Sentences to Candidate Distributions
Basics
How can one System One multimodal readout produce a full ranking?
Multimodal Reranking Is a System One Decision
Show answer and example
Answer: Encode query and first-stage candidates in one context and read their label probabilities at the first answer position, then sort. The unprocessed tail stays in retrieval order. No ranking text is generated, but inputs still need encoding.
Illustrative example: Example: Read A/B/C probabilities for top images and sort them, leaving later candidates in their original order.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Core mechanism
Why correct by the probability of at least one draw after deduplicating SSA samples?
Stratified Sparse Attention: Reducing the Sparse-to-Dense Gap at Extreme Sparsity
Show answer and example
Answer: A repeated token appears only once in the set, so inclusion is 1−(1−p)^m rather than single-draw p. Deterministically retained heavy items have inclusion one. The final self-normalized ratio is not strictly unbiased.
Illustrative example: At p=0.2 and two draws, weight one retained row using 0.36, not 0.2.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Implementation details
How does SSA compute small inclusion probabilities and test full budget?
Stratified Sparse Attention: Reducing the Sparse-to-Dense Gap at Extreme Sparsity
Show answer and example
Answer: Use −expm1(m·log1p(−p)) and log-space normalization for stability. Return dense attention explicitly at k=N. Detached discrete allocation is not a gradient path.
Illustrative example: A budget of all keys must match dense output without randomly losing tail tokens.
Sections in the Chinese source: 核心公式, 验证与边界. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Method comparison
Why not correct deduplicated with-replacement samples by 1/p_i?
Stratified Sparse Attention: Reducing the Sparse-to-Dense Gap at Extreme Sparsity
Show answer and example
Answer: Each token enters once with π_i=1−(1−p_i)^m. Divide by inclusion, not per-draw probability. The self-normalized numerator/denominator ratio still need not be unbiased.
Illustrative example: A token drawn three times but retained once is not three independent weighted observations.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Basics
Why retain heavy entries deterministically and sample the tail?
Stratified Sparse Attention: Reducing the Sparse-to-Dense Gap at Extreme Sparsity
Show answer and example
Answer: Top-k drops the tail; uniform sampling can waste budget. SSA uses proxy scores to divide deterministic and residual budgets, then corrects by inclusion probability.
Illustrative example: Strong evidence remains while weaker entries have a chance of inclusion.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Core mechanism
Why keep original RoPE frequencies and head scaling after RACE narrows attention?
RACE: RoPE-Aligned Attention Compression Based on Query-Key Activation Energies
Show answer and example
Answer: This is a subspace of an existing model. Preserve complete matched Q/K rotation pairs, original frequencies and width scaling; fold V projections into weights for native low-dimensional compute. Calibration energy optimizes a bound, not global LM loss.
Illustrative example: Keep original second and fifth frequencies rather than renumbering them first and second.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Implementation details
Why fold W_O when compressing V?
RACE: RoPE-Aligned Attention Compression Based on Query-Key Activation Energies
Show answer and example
Answer: W_V′=W_VP_V and W_O′=P_VᵀW_O define one subspace path. Changing V alone mismatches shape or function. Detach calibration and SVD while retaining gradients through folded forward.
Illustrative example: A four-dimensional V needs a matching output map, not the original eight-dimensional interpretation.
Sections in the Chinese source: 核心公式, 本地复现. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Method comparison
Why compress Q/K in pairs but V with an SVD subspace?
RACE: RoPE-Aligned Attention Compression Based on Query-Key Activation Energies
Show answer and example
Answer: Q/K preserve common RoPE planes and matching geometry. Values carry aggregated content and can use a calibrated right singular space. One common pruning rule changes their roles.
Illustrative example: Keep the corresponding K pair for each Q pair; V need not use those coordinates.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Basics
Why directly reduce native attention head width?
RACE: RoPE-Aligned Attention Compression Based on Query-Key Activation Energies
Show answer and example
Answer: Low-rank storage followed by full reconstruction saves memory but adds work. RACE folds selections and V subspaces into projections so attention runs on retained dimensions.
Illustrative example: Run four-dimensional attention directly instead of rebuilding eight dimensions.
Sections in the Chinese source: 背景与主要改动, 本地复现. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Core mechanism
Why not pull all LocaLeJEPA clusters toward one Gaussian?
LocaLeJEPA: Maximum Entropy Representations under Geometric Priors
Show answer and example
Answer: Local clusters have different geometries and positions. A global pull can destroy layout. Fix group assignments, recenter residuals per cluster, and match a uniform solid-ball distribution, with an explicit small-cluster fallback.
Illustrative example: Two clusters for different motion modes can keep distinct centers while each has a regular local shape.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Implementation details
How do small-cluster filtering and cluster count change regularization strength?
LocaLeJEPA: Maximum Entropy Representations under Geometric Priors
Show answer and example
Answer: Keep clusters with at least four samples; fall back to the full batch and log it if none qualify. Multiply each cluster statistic by sample count, then average equally over valid clusters and views. Count changes both granularity and scale. Integrate the ball characteristic function, not a Gaussian target.
Illustrative example: Splitting into more valid small clusters changes the value through weighting and averaging even with the same λ.
Sections in the Chinese source: 核心公式, 本地复现, 验证与边界. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Method comparison
How does a uniform solid ball differ from a Gaussian or a unit sphere surface?
LocaLeJEPA: Maximum Entropy Representations under Geometric Priors
Show answer and example
Answer: The solid ball puts mass inside radius √(d+2), giving identity covariance. A Gaussian is unbounded and a sphere surface has mass at a fixed radius only. Their characteristic functions differ. Gaussian formulas or vector normalization cannot replace the target.
Illustrative example: A uniform disk includes points near its center; a circle does not, and a Gaussian extends to arbitrary radius.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Basics
Why prevent collapse if two views of the same image should agree?
LocaLeJEPA: Maximum Entropy Representations under Geometric Priors
Show answer and example
Answer: A constant vector also gives perfect view agreement. LocaLeJEPA wants invariance across augmentations and distinguishable representations across inputs while respecting local geometry. A low agreement loss does not establish useful features.
Illustrative example: Cat crops should be close, but collapsing cats and cars together ruins retrieval.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Core mechanism
Why doesn't inheriting a parent harness method mean running several ancestor agents?
Object-Oriented Harness: Evolving Agent Harnesses through Object-Oriented Design
Show answer and example
Answer: Inheritance reuses code: one instance invokes a non-overridden parent method, without separate ancestor trajectories. A subclass changes only needed hooks; deployment selects one frozen class. Cluster gains do not guarantee global gains, and self-confidence ablations are not calibrated success probability.
Illustrative example: Example: Override the search prompt while retaining the parent's terminal implementation. That is one agent, not two parallel collaborators.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Implementation details
Why must a specialized class start in the same initial environment, whereas selecting Base can reuse its result?
Object-Oriented Harness: Evolving Agent Harnesses through Object-Oriented Design
Show answer and example
Answer: Base runs before routing, so its existing result can be used if it is ultimately selected. A specialized class must reset through tool_factory so that it does not inherit Base's progress. Real external effects are not guaranteed to roll back. Trusted local Python classes and allowlisted data isolation are not a security sandbox, and a scripted actor does not demonstrate LLM capability.
Illustrative example: If Base has already created half the files, letting a specialist continue would make it look more successful. A fair comparison starts from the same initial file state.
Sections in the Chinese source: 核心公式, 本地复现, 成本与复现边界. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Method comparison
What is the trade-off between strict target-cluster admission and one global mean?
Object-Oriented Harness: Evolving Agent Harnesses through Object-Oriented Design
Show answer and example
Answer: The target cluster must strictly beat its current specialist and pass extra checks, so a global mean cannot hide a local regression. This does not guarantee gains elsewhere. Routing uses public tasks and observed base traces, not test grader scores. Self-reported 0.7 is uncalibrated.
Illustrative example: Example: Validate a spreadsheet-specific change on spreadsheets. Gains on an easier unrelated cluster cannot mask no improvement there.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Basics
Why use inheritable harness classes for specialization across tasks?
Object-Oriented Harness: Evolving Agent Harnesses through Object-Oriented Design
Show answer and example
Answer: A generic flow may not suit every task, while copying whole implementations loses shared verified logic. Classes express reusable behavior and overrides, then route tasks to suitable variants. Inheritance organizes code and does not add runtime agents or guarantee universal improvement.
Illustrative example: Example: A browsing subclass reuses recovery and changes search. Selecting it executes one flow, not parent and child separately.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why can the same final text prefix correspond to different, nonreusable KV caches?
Show answer and example
Answer: KV depends on the history and positions visible when it was computed. Dropping tokens at different times can alter later states. repos must also transform RoPE keys for their new positions; updating indices alone or rotating values is wrong. Cache identity must therefore track position versions and operation history, not just logical text.
Illustrative example: Example: Evicting a tool response before generation versus afterward changes what later tokens could see. Matching final retained text does not make those caches interchangeable.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
Why does repos rotate keys but not apply the same RoPE transform to values?
Show answer and example
Answer: Moving a position from p to q adjusts an already rotated key by R(q)R(p)^-1, while the value stays unchanged. drop itself does not renumber. The cache system also needs operation history and versions. The local run does not implement AKV or serving; a few tensor rotations are not full cache engineering.
Illustrative example: Example: Moving a key from position ten to five undoes the old rotation and applies the new one. Rotating its value would change how its content is read.
Sections in the Chinese source: 核心公式, 本地复现, 成本与复现边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
How does direct KV management differ from summarizing text for recovery and reuse?
Show answer and example
Answer: Summarization changes logical input and can invalidate later KV, requiring fresh prefill. Direct management preserves logical history but changes active state and position versions. Different operation timing can produce different KV for the same final text and token set, so cache reuse identity must include history.
Illustrative example: Example: Dropping tool KV before versus after generating a sentence can change that sentence's hidden state even when the final retained text is identical.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Why can a long-running agent keep its chat history while running out of inference cache memory?
Show answer and example
Answer: Logical conversation stores text, while KV cache stores each layer's intermediate representations of past tokens. Their purpose and footprint differ. Longer conversation increases cache memory and subsequent reads. Cache Engineering manages that state directly while preserving logical history, but recovering evicted cache still costs computation.
Illustrative example: Example: Ten thousand tool-log lines can remain in the conversation even while their KV is inactive. Reusing those logs later can require recomputing representations.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Core mechanism
Why retain a temporarily low-scoring child interface repository in the population rather than keeping only changes that strictly improve on their parents?
Show answer and example
Answer: Evolutionary search may have to pass through local valleys or combine different changes. Retaining descendants lets later mutation and crossover reuse their structure. Validation scores guide parent selection, but validation traces and labels are not exposed to the Meta-Agent. What evolves is the action and observation interface itself; the search is not greedy acceptance of the best patch in each round.
Illustrative example: An interface change may perform modestly on its own but work well with observation compression from another lineage. Immediately discarding all low-scoring children eliminates that opportunity.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Implementation details
Why does BuilderContext omit validation diagnostics, and why does low-score pruning retain lineage?
Show answer and example
Answer: Validation is used only to select parents. Passing diagnostics would turn a protected selection set into development feedback. Pruning revokes parent eligibility while retaining audit history and final candidates, including failures. The local controller uses trusted callbacks and synthetic tasks; it neither executes arbitrary interface source nor demonstrates improvement in a real GUI.
Illustrative example: If the Meta-Agent sees filenames from validation failures and fixes them directly, it is no longer developing from the permitted training traces. Deleting low-scoring records would also conceal the cost of search.
Sections in the Chinese source: 核心公式, 本地复现, 成本与复现边界. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Method comparison
How do single-parent mutation and two-parent crossover differ in where they get their changes?
Show answer and example
Answer: Mutation modifies the interface and usage knowledge of a single parent. Crossover combines structures or experience from two different parents. Both construct candidates from unlabeled training traces; private validation outcomes are used only for population selection, not handed to the Meta-Agent as answers. Crossover broadens the available combinations but can introduce incompatibilities, so freeze the resulting child before independently evaluating it.
Illustrative example: One parent improves screenshot observations and another improves bulk editing. Crossover can attempt both, whereas single-parent mutation works from just one current repository.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Basics
How does evolving the interface differ from accumulating skills composed from existing tools?
Show answer and example
Answer: Skills generally combine procedures over a fixed action and observation interface. S4 lets a Meta-Agent change interface code and usage knowledge, changing what the task agent can see and do. The task model and harness are held fixed to isolate the interface effect, but a new interface still needs execution on real tasks; renaming the repository version is not enough.
Illustrative example: Replacing cell-by-cell editing with bulk operations over a region changes the interface. Recording that cells should be sorted before editing is usage knowledge for the existing tool.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why allow collisions rather than shifting old codes?
Show answer and example
Answer: The generator already assigns probability to old paths. Remapping sends it to other objects and short retraining may not repair it. Expand collisions while preserving addresses; test narrow-beam stability.
Illustrative example: Keep old A pointing to its item and expand A for a newcomer rather than move the original to B.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
Can old items shift when new items collide with exclusive tokens?
Show answer and example
Answer: No. Freeze all old SIDs, expand permitted collisions, and declare tail capacity with failure on exhaustion. Preserve absent old items; no silent modulo or re-encoding.
Illustrative example: A new head item should not push old codes down one slot.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
Why doesn't acceptable wide-beam retrieval prove remapping was harmless?
Show answer and example
Answer: Wide search can conceal rank/path drift visible under narrow serving budgets. Separate mapping and training effects and check path stability.
Illustrative example: A product found deep in a wide beam may be missing from top paths.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why audit catalog updates separately from continued training?
Show answer and example
Answer: Retrieval learns query-to-SID semantics. Remapping old items changes learned paths, and continued training may not repair arbitrary reassignments.
Illustrative example: A familiar code for shoes becomes wrong if silently remapped to bags.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why does MAPS's Reflector separate profile errors from script errors and route a minimal edit?
Show answer and example
Answer: Poor feedback can arise from misunderstanding the user or from messaging and factual mistakes. Rebuilding everything wastes compute and can damage correct stages. Role attribution targets the relevant stage and keeps profile, script, and video generation auditable. Mixed rewards cover structure, personalization, and factuality. Large models and private environments make this review-only; there is no keyword substitute.
Illustrative example: Illustration: Fix a script that overstates product performance when the profile is right. If the gift recipient was misunderstood, polishing shots cannot fix targeting.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Why doesn't MAPS perform effective RL when group feedback never differs?
Show answer and example
Answer: Scripts in the same context need different rewards. Identical rewards yield zero centered advantage and no preference direction. The actual judge and video pipeline also matter. The source explicitly avoids a keyword proxy; evaluating the formula alone does not reproduce training or the environment.
Illustrative example: Illustration: If a crude rule gives all four scripts full marks, an optimizer call cannot teach which is more personalized or establish improved four-agent video creation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
What does MAPS's mixed reward measure that keyword counts cannot?
Show answer and example
Answer: Structure includes scene roles and physical plausibility; semantic judges score personalization and product consistency. Keyword coverage cannot detect contradictions or user fit. A proxy can test interfaces but is not algorithm evidence. Reward weights do not guarantee judge quality, and text scores differ from creative outcomes.
Illustrative example: Illustration: 'Durable' and 'perfect for you' can satisfy keywords while calling ordinary shoes fireproof. Product consistency must check facts.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why does MAPS split creative generation into Profile, Scripter, Director, and Reflector?
Show answer and example
Answer: The roles extract needs and style, write product messaging and scenes, execute video, and route feedback. This localizes errors rather than rebuilding all stages. Role names do not confer capability; real semantic models, video generation, and a credible environment are needed. The local scope is review-only for cost.
Illustrative example: Illustration: Fix copy that overclaims, or fix video execution that violates the script, rather than forcing the profiling agent to rewrite a correct preference.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Core mechanism
Why accept only teacher corrections matching the visual contrast sign?
Reinforcement Learning with Visually Attributable On-Policy Self-Distillation
Show answer and example
Answer: Privileged crop changes may be stylistic. Require compatible evidence direction and use bounded modulation of verifier advantage. Only visual input changes in same EMA model; arbitrary text logits and local documentation do not reproduce the chain.
Illustrative example: If preference changes after privileged crop but not after occluding evidence, don't automatically treat it as visual guidance.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Implementation details
Can VAG's teacher branches be different models, and was this run?
Reinforcement Learning with Visually Attributable On-Policy Self-Distillation
Show answer and example
Answer: Not under the same protocol: one EMA teacher, only visual input changes. The local page describes equations without executing multimodal training. Keep strict sign agreement and verifier advantage.
Illustrative example: Comparing a large vision teacher to a small text model confounds architecture and evidence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Method comparison
How does VAG differ from fixed textual teacher logits?
Reinforcement Learning with Visually Attributable On-Policy Self-Distillation
Show answer and example
Answer: The definition needs the same EMA vision model under real paired views. Fixed logits only simulate the formula, not visual sensitivity or attribution.
Illustrative example: Two manually supplied scores aren't an occlusion experiment.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Basics
Why compare evidence-preserving and degraded crops in VAG?
Reinforcement Learning with Visually Attributable On-Policy Self-Distillation
Show answer and example
Answer: Privileged corrections can mix visual evidence with style. Contrast matched visual conditions to estimate direction, retain agreeing corrections and bound modulation of verifier advantage.
Illustrative example: After cropping, check whether a teacher's revision follows changed visual evidence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Core mechanism
Why freeze only APSD teacher rather than detach the interpolated/extrapolated target?
Correctness-Dependent Asymmetric Target Distributions for Self-Distillation Policy Optimization
Show answer and example
Answer: Target depends on current student as well as teacher. Detach loses the student coupling that differs for success/failure. Forward equality doesn't imply backward equality. Feedback is privileged training context only.
Illustrative example: Identical target probabilities can yield different parameter updates when only one graph retains target dependence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Implementation details
Which PoE tensors retain APSD gradients, and how interpret default α?
Correctness-Dependent Asymmetric Target Distributions for Self-Distillation Policy Optimization
Show answer and example
Answer: Student and its target branch are differentiable, teacher detached. Use log space to avoid underflow. Correct α is between zero and one, error α above one. Local RLRF settings differ from paper RLVR.
Illustrative example: Local 0.5 and 1.5 aren't universal paper settings.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Method comparison
Why isn't freezing APSD self-teacher the same as detaching its entire target?
Correctness-Dependent Asymmetric Target Distributions for Self-Distillation Policy Optimization
Show answer and example
Answer: Teacher q is frozen, but product-of-experts target also depends differentiably on p. Whole-target detach removes a defining coupling term, despite identical forward values.
Illustrative example: Detaching a target branch partly built from current student changes optimization, not merely memory.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Basics
Why interpolate APSD successes but extrapolate beyond teacher for failures?
Correctness-Dependent Asymmetric Target Distributions for Self-Distillation Policy Optimization
Show answer and example
Answer: Success should retain existing student preferences with gentle interpolation. Failures use stronger teacher-direction correction. Feedback selects a target regime, not guaranteed correctness.
Illustrative example: Use lighter changes after a correct response and stronger correction after failure.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Core mechanism
Does KronLSTM update a full value vector or one scalar key–value association?
Kronecker Delta Rule for Long Context Linear Attention
Show answer and example
Answer: It uses vᵀSk and updates along vkᵀ, unlike correcting an entire vector error. Interference depends on overlap in both key and value; orthogonality on either side blocks that update's interference. Forget, erase, and write are separate; do not multiply the write coefficient by the erase gate.
Illustrative example: Similar keys with orthogonal values do not interfere under one scalar association update, without guaranteeing no future interference.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Implementation details
Why not copy ordinary delta update's forget direction and write coefficient?
Kronecker Delta Rule for Long Context Linear Attention
Show answer and example
Answer: Forgetting acts on the right along the key axis. The decoupled write is c·vkᵀ, not βc·vkᵀ; only erase uses β. Compare an independent vec reference instead of checking loss alone.
Illustrative example: Small β should not suppress the independent write gate c.
Sections in the Chinese source: 核心公式, 实现与运行. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Method comparison
How does KronLSTM's error differ from DeltaNet for interference analysis?
Kronecker Delta Rule for Long Context Linear Attention
Show answer and example
Answer: DeltaNet uses Sk−v, while KronLSTM uses scalar vᵀSk−c. Interference involves both key and value inner products; one orthogonal side removes interference from this write, not all future forgetting.
Illustrative example: Similar keys with orthogonal values can avoid interference despite key overlap.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Basics
What memory quantity does KronLSTM correct?
Kronecker Delta Rule for Long Context Linear Attention
Show answer and example
Answer: It corrects scalar association vᵀSk rather than forcing the entire retrieved vector to equal v. The update follows vkᵀ with separate forget and erase/write gates.
Illustrative example: Adjusting one association need not rewrite every value direction for that key.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Core mechanism
Why isn't FLINT score a symmetric mean similarity at fixed V?
FLINT: Fixed-budget Late Interaction
Show answer and example
Answer: Learned latents read tokens on both sides. Score each query latent by its best document match, then sum across queries. Fixed budgets do not make pooling arbitrary or scoring symmetric.
Illustrative example: Location and time query latents each find evidence; reversing which side is summed generally changes the score.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Implementation details
Why define FLINT padding masks and fully masked inputs explicitly?
FLINT: Fixed-budget Late Interaction
Show answer and example
Answer: Padding cannot participate in cross-attention; local True denotes valid tokens and wholly masked rows fail. Preserve pre-LN and both residual stages, not a bare attention pool.
Illustrative example: Padding half a batch must not change what latents read.
Sections in the Chinese source: 核心公式, 本地复现. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Method comparison
How does latent pooling differ from truncation or averaging?
FLINT: Fixed-budget Late Interaction
Show answer and example
Answer: Cross-attention reads all valid tokens before residuals and FFN, producing task-relevant fixed-budget vectors. Truncation loses tails and means use different aggregation.
Illustrative example: A latent can attend to a fact at the end that first-four truncation never sees.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Basics
Why compress both query and document to a fixed vector count?
FLINT: Fixed-budget Late Interaction
Show answer and example
Answer: Token-level multivector indexing and search grow with length. Separate learned latents produce fixed V vectors per side for explicit cost control.
Illustrative example: Long and short documents can both yield four vectors, as can queries.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Core mechanism
Why fix prediction order and isolate gold in grouped DeepSEA reporting?
DeepSEA: A Multilingual Evaluation Benchmark for Southeast Asia
Show answer and example
Answer: Validate language, size, and order so predictions align to their examples. Expose only public task fields; keep answers and annotations with the evaluator. This evaluates lower-resource languages, not model training.
Illustrative example: Swapped outputs across languages become alignment errors; the predictor must not see references.
Sections in the Chinese source: 背景与主要改动, 核心公式与评分口径. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Implementation details
Should prediction callbacks get the evaluator or benchmark file and promise not to read gold?
DeepSEA: A Multilingual Evaluation Benchmark for Southeast Asia
Show answer and example
Answer: No. Use allowlisted immutable public input and separate deep-copied private references. Reject missing languages, bad counts, or nonstring outputs. This is data-flow isolation, not an OS sandbox; do not add shell bypasses.
Illustrative example: Provide question, options, and context, not raw answer_with_cot records.
Sections in the Chinese source: 公共输入与 Gold 隔离. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Method comparison
Are deterministic scorers and LLM judges interchangeable in DeepSEA?
DeepSEA: A Multilingual Evaluation Benchmark for Southeast Asia
Show answer and example
Answer: Follow the fixed upstream protocol for each task. Substituting judges changes comparability. Bind outputs to language, subject, and row; group summaries do not fix misalignment.
Illustrative example: Pairing language A row one with a language B answer invalidates scoring even if row counts match.
Sections in the Chinese source: 背景与主要改动, 公共输入与 Gold 隔离. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Basics
Why not report only one DeepSEA multilingual average?
DeepSEA: A Multilingual Evaluation Benchmark for Southeast Asia
Show answer and example
Answer: Language-resource strata and tasks expose gaps hidden by a mean. The project provides data and evaluation protocol, not a parameter-training algorithm.
Illustrative example: High-resource successes can mask low-resource failures.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why use one memory-worth model for both retrieval and eviction rather than rank only by similarity?
Remembering Has a Price: Learning Memory Worth under Resource Constraints
Show answer and example
Answer: Similar memories may be long, obsolete, or unreliable. Under budget, compare actual task value. Stable base features supply interpretable signals, and RFM learns a query-dependent residual. The same value scale guides retrieval and retention. Feedback updates only memories actually selected and used; do not invent outcomes for unvisited entries.
Illustrative example: Example: Two memories concern the same interface. The old one is more similar but obsolete; the newer one is shorter and reliable, and may be worth retaining instead.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
Why can a newly admitted memory still be evicted immediately?
Remembering Has a Price: Learning Memory Worth under Resource Constraints
Show answer and example
Answer: Admission checks novelty or probe value relative to the best old memory. The expanded library then evicts the lowest overall value, and the new item competes too. Probes use real current and historical queries, not future test questions. Only selected pairs receive updates, not invented counterfactual success labels.
Illustrative example: Example: A sufficiently novel candidate enters a full library but ranks last by shared value. It should leave rather than force out a stronger existing memory.
Sections in the Chinese source: 核心公式, 本地复现与边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
Why separate fast residual fitting from slow feature-geometry updates?
Remembering Has a Price: Learning Memory Worth under Resource Constraints
Show answer and example
Answer: Fast updates fit value discrepancies from new feedback promptly. Slow updates use gradient statistics to adapt kernel-distance geometry without large representation swings after every observation. These are different adaptation timescales. The residual model cannot see all candidate outcomes; only used memory pairs have real labels.
Illustrative example: Example: One failure can lower a memory's residual worth. Multiple feedback observations can later change which feature differences the kernel amplifies.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Should the most related memories always be kept, and what resource limits does TokenWorth face?
Remembering Has a Price: Learning Memory Worth under Resource Constraints
Show answer and example
Answer: Memory capacity, interactions, and generation length are finite. A long, superficially relevant memory can crowd out more useful information. TokenWorth treats retention and use as value allocation under resource competition rather than maximizing library size or similarity alone.
Illustrative example: Example: A long guide repeats known steps, while a short note records a key permission restriction. The shorter note can deserve a slot despite weaker word overlap.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Core mechanism
Why protect full-action probabilities instead of token votes or masking the whole success?
Show answer and example
Answer: A specific action in a success is protected when teacher supports it less; probability combines all its tokens. Cancel distillation there and add a soft floor, leaving other actions learnable. Shared parameter changes can still erode protected actions.
Illustrative example: Teacher can like some tool-call tokens and dislike others, so token votes can oppose the whole-call probability.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Implementation details
Does SPOD compare temperature sampling probability or raw full-action probability?
Show answer and example
Answer: Use raw pre-temperature/pre-constraint probabilities including termination, sum token logp. Recompute gate pre-update but detach it; average by action, state and episode so long traces don't dominate.
Illustrative example: Mean token probability isn't full-action probability and changes protection by length.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Method comparison
How does a SPOD soft floor differ from masking an entire successful episode?
Show answer and example
Answer: Only selected actions are protected; other decisions still distill. A soft penalty below threshold isn't a hard guarantee and shared updates can change protected behavior.
Illustrative example: One useful successful decision need not block all teacher learning in its episode.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Basics
Is a SPOD decision one token or an action of multiple tokens?
Show answer and example
Answer: Protect a full state-conditioned action including native termination, not one token or an episode. Labels are episode-level but protection is decision-specific.
Illustrative example: Tool name, arguments and terminator form one action within a possibly multi-action success.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Core mechanism
Why stack related matrices into a tensor instead of flattening them for ordinary Muon?
T-MUON: Tensorizing Muon via t-SVD for Large Language Model Pre-training
Show answer and example
Answer: Transform along the third axis, compute polar directions slice-wise, and invert the transform. This couples related matrices. Grouping and transform are separate choices. Restore original matrix scaling and zero rank-zero slices to avoid spurious updates.
Illustrative example: Q/K/V transform-domain slices are not equivalent to one flattened matrix.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Implementation details
Should fan scaling use head slices or original matrices?
T-MUON: Tensorizing Muon via t-SVD for Large Language Model Pre-training
Show answer and example
Answer: Use original fan_out/fan_in after reconstruction. Rank-zero slices must yield zero instead of arbitrary thin-SVD nullspace updates. Exact local SVD does not establish production polar throughput.
Illustrative example: Even after slicing by head, scale using the unsplit dimensions.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Method comparison
Why ablate grouping and transforms separately?
T-MUON: Tensorizing Muon via t-SVD for Large Language Model Pre-training
Show answer and example
Answer: Grouping decides which matrices share a tensor; the transform decides how slices mix. Changing both confounds causes.
Illustrative example: QKV with DCT versus heads with DFT does not isolate DFT.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Basics
Why put related matrices on the third tensor axis?
T-MUON: Tensorizing Muon via t-SVD for Large Language Model Pre-training
Show answer and example
Answer: It lets layers, QKV, or heads interact through a third-axis transform before slice-wise polar and inverse mapping. Both grouping and transform define the method.
Illustrative example: Stacking Q/K/V couples them instead of handling three unrelated matrices.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why shouldn't an archive refresh reset the virtual queue and global round count?
MemPrice: Online Memory Allocation via Adaptive Pricing for Long-Horizon Agents
Show answer and example
Answer: The queue represents cumulative resource pressure, and the round count scales the dynamic price. Refresh only extracts a new representation from an external archive. Resetting both would forget past overuse and suddenly underprice detail. Hard capacity still applies every round; an average occupancy target cannot replace it.
Illustrative example: Example: After several nearly full context windows, a summary refresh should not zero the price and cause another wave of expensive full-text retention.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
How does MemPrice fit each memory's alternatives into a hard token budget?
MemPrice: Online Memory Allocation via Adaptive Pricing for Long-Horizon Agents
Show answer and example
Answer: Enumerate full text, summary, abstraction, keywords, and deletion, then measure sizes with the tokenizer. Use multiple-choice knapsack to choose at most one representation per memory. Each round must stay within B. The average occupancy target changes price, not whether the current window may exceed capacity. Archive refresh does not invent deleted detail.
Illustrative example: Example: With one hundred tokens left and two eighty-token originals, select summaries if useful instead of combining the independently best originals.
Sections in the Chinese source: 核心公式, 本地复现与边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
How does dynamic pricing represent long-run capacity pressure that relevance Top-K misses?
MemPrice: Online Memory Allocation via Adaptive Pricing for Long-Horizon Agents
Show answer and example
Answer: Top-K uses only the current ranking. Dynamic price comes from a virtual queue of persistent occupancy, so historical overuse makes retaining detail more expensive. Hard capacity caps one round; average occupancy controls long-run pressure. They are not the same threshold, and the mechanism does not guarantee global optimality for all state paths.
Illustrative example: Example: Repeatedly filling the window raises the price and encourages lower fidelity. A currently relevant memory may no longer justify keeping a long original.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Why are MemPrice's five representation levels a joint allocation problem?
MemPrice: Online Memory Allocation via Adaptive Pricing for Long-Horizon Agents
Show answer and example
Answer: Each memory can be full text, summary, abstraction, keywords, or deletion. These differ in size and usefulness and compete for one hard token budget. Independently picking the best version can exceed capacity. Optimize jointly while choosing at most one per item; deleted detail cannot be reconstructed from nothing in the current window.
Illustrative example: Example: One full-text memory can crowd out two summaries. Compare the whole group instead of choosing highest fidelity independently for every item.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Core mechanism
Why is three local layers followed by one global layer different from shortening every context window?
Muse Glimmer 30B: An On-Device Agentic Model
Show answer and example
Answer: Local layers limit most interactions, while global layers still exchange distant information. GQA separately reduces KV heads. These are architectural trade-offs, not reproduction of distillation, post-training, or device performance.
Illustrative example: A distant heading can affect the current paragraph in a global layer, while local layers process adjacent sentences.
Sections in the Chinese source: 背景与主要改动, 核心公式与结构. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Implementation details
Can the 52 layers share one attention and RoPE template?
Muse Glimmer 30B: An On-Device Agentic Model
Show answer and example
Answer: No. Repeat [Local, Local, Local, Global] 13 times with 32 query heads and two KV heads. Only local layers apply RoPE. The vision branch separately compresses 1024 patches to 256 tokens via 2×2 grouping. One layer is not the 29.6B model.
Illustrative example: Applying RoPE everywhere or alternating local and global changes the architecture even with 52 layers.
Sections in the Chinese source: 核心公式与结构, 本地复现与成本判断. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Method comparison
How does late-stage cross-entropy differ from fitting truncated teacher logits?
Muse Glimmer 30B: An On-Device Agentic Model
Show answer and example
Answer: Distillation uses teacher predictions, but top-k truncation drops the tail. Late cross-entropy directly supervises the target token; the paper connects the switch to truncation bias. It is not correct to describe the entire recipe as teacher matching.
Illustrative example: CE still uses a target token even if the saved teacher top-k omitted it.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Basics
Why count weights, KV, vision, and drafter memory together for Muse Glimmer?
Muse Glimmer 30B: An On-Device Agentic Model
Show answer and example
Answer: Device memory must hold the entire inference process. Long agent context grows KV, and vision and speculative heads add memory. A quantized language model under a size cap does not alone fit a 32GB device. Use real inputs and deployment settings.
Illustrative example: Weights below 20 GB may still leave too little space for long-trajectory KV and temporary vision tensors.
Sections in the Chinese source: 背景与主要改动, 核心公式与结构. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Core mechanism
Why retain retrieval-program supervision for a wrong answer while masking its answer span?
Learning to Program Retrievals: Composing Search Primitives via Reinforcement Learning
Show answer and example
Answer: A syntactically valid, executed program can contain useful retrieval structure even when the final answer is wrong. Mask wrong-answer supervision so it does not teach generation errors. Actors cannot see verifier gold, and environment observations are not policy-generated tokens.
Illustrative example: Example: A program combines and deduplicates two retrieval routes correctly but the answerer copies a date incorrectly. Keep program supervision and mask the wrong date.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Implementation details
How does the local restricted RPL program control retrieval, fusion, and final output?
Learning to Program Retrievals: Composing Search Primitives via Reinforcement Learning
Show answer and example
Answer: It allows at most four distinct queries, prescribed retrieval and reranking widths, real fusion of multi-route results, and a cap on printed documents. Conflicting contents for one ID are rejected, and reranking cannot add unretrieved documents. Local execution interprets data classes and an injected SDK, not arbitrary Python or the paper's token limits.
Illustrative example: Example: Retrieving twenty items and printing three legitimately separates compute and observation. Quietly injecting an unretrieved document during reranking breaks provenance.
Sections in the Chinese source: 核心公式, 本地复现, 成本与复现边界. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Method comparison
How does mutating retrieval structure while preserving the query differ from asking the model for a new query?
Learning to Program Retrievals: Composing Search Primitives via Reinforcement Learning
Show answer and example
Answer: Keeping the query isolates retrieval mode, width, fusion, and reranking changes. A new query changes semantics along with control flow. Local mutations can restrict exploration, but using gold to produce correct documents is not a valid mutation.
Illustrative example: Example: Widen retrieval from three to five under the same query to test evidence coverage. Adding answer keywords simultaneously confounds the change.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Basics
Why does programmable retrieval return only explicitly printed documents to the model?
Learning to Program Retrievals: Composing Search Primitives via Reinforcement Learning
Show answer and example
Answer: A program can retrieve, fuse, deduplicate, and rerank a large pool, but context receives only explicit outputs. This separates retrieval work from observation budget. Secretly appending intermediates bypasses program selection and invalidates the policy comparison.
Illustrative example: Example: Retrieve 50 documents and print three after reranking. The model sees only those three, not the other 47.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Core mechanism
Why can't high computational-proxy scores replace real experimental feedback, or be compared across rounds as fixed-test gains?
Aptamers as a sandbox for lab-in-the-loop agentic molecular discovery
Show answer and example
Answer: Computational scores can disagree with physical measurements. The point of the loop is that new measurements can refute predictions and preserve evidence of failures. Candidates and strategies differ across rounds, and hit rates depend on the selected distribution, so these are not directly comparable machine-learning scores on the same test set. The work assesses research orchestration and does not imply specific laboratory procedures.
Illustrative example: If one round tests only the proxy's favorite candidates and the next focuses on exploration, different hit rates may reflect selection rather than improved generalization.
Sections in the Chinese source: 背景与主要改动, 核心机制与评测. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Implementation details
Why doesn't the local record replace the physical endpoint with simulated measurements or predicted scores?
Aptamers as a sandbox for lab-in-the-loop agentic molecular discovery
Show answer and example
Answer: Real experiments can falsify the computational proxy, a defining feature that would disappear if they were replaced. The local entry documents research orchestration and cost only; it includes no model, GPU receipt, capability metric, or Evolve integration. A numeric fixture cannot stand in for a specialized laboratory or feedback that takes days, and no experimental operating procedure is provided.
Illustrative example: Randomly generating a 'measurement passed' label can test a messaging interface, but does not establish physical efficacy or complete an experimental loop.
Sections in the Chinese source: 核心机制与评测, 本地复现与成本决策. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Method comparison
What information is added by a new physical measurement versus repeated reanalysis of existing experimental records?
Aptamers as a sandbox for lab-in-the-loop agentic molecular discovery
Show answer and example
Answer: Reanalysis may improve interpretation or candidate selection but creates no new physical observation. A new measurement tests a previously unverified candidate and may expose model omissions. Both can support research, but recomputation is not an independent experiment and a simulated proxy cannot replace the experimental endpoint. For cross-round comparisons, preserve changes in candidates and measurement conditions.
Illustrative example: Reranking ten existing results still leaves ten measurements. Testing an eleventh candidate adds a genuinely new piece of external feedback.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Basics
Why should aptamer design repeatedly connect computational proposals to real experimental feedback?
Aptamers as a sandbox for lab-in-the-loop agentic molecular discovery
Show answer and example
Answer: A computational model estimates candidate properties from existing data; actual molecular behavior may depend on factors it did not model. This work uses real measurements to test proposals and inform the next round, making experimentation rather than a predicted score the decisive feedback. Changing candidates and selection conditions across rounds means progress in the loop does not automatically demonstrate stronger performance on a fixed test set.
Illustrative example: If a candidate favored by the model performs modestly when measured, use the counterexample in the next round. Raising its predicted score does not alter the physical measurement.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Core mechanism
Why convert backed-up tree credit into a bounded Bernoulli target?
Tree-Aware Posterior Distillation for Long-Horizon Agentic Reinforcement Learning
Show answer and example
Answer: Rewards set direction but no stopping distance. A fixed finite target stops or reverses gradients when reached or exceeded. Deduplicate shared turns and weight branches by full actor-span likelihood, not average token score.
Illustrative example: Repeatedly training one success group stops when it has learned enough instead of reinforcing indefinitely.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Implementation details
Is TreeOPSD's Bernoulli a full next-token teacher?
Tree-Aware Posterior Distillation for Long-Horizon Agentic Reinforcement Learning
Show answer and example
Answer: No. A two-point posterior describes a sampled turn relative to old policy. Use full actor-span likelihood and leaf-reward sample SD, exclude observations, and retain bounded-target derivation.
Illustrative example: The two outcomes are not two vocabulary tokens.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Method comparison
How does bounded Bernoulli teaching differ from repeatedly applying fixed advantage?
Tree-Aware Posterior Distillation for Long-Horizon Agentic Reinforcement Learning
Show answer and example
Answer: Frozen posterior gives a finite target: gradient stops at the desired relative change and reverses beyond it. Fixed advantages can keep pushing under repeated updates.
Illustrative example: After sufficient adjustment, stop rather than drive the action farther on the same batch.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Basics
Why organize TreeOPSD shared prefixes as a tree?
Tree-Aware Posterior Distillation for Long-Horizon Agentic Reinforcement Learning
Show answer and example
Answer: Different continuations from one history reveal outcomes. Back up leaf rewards with behavior likelihood into local credit and count shared turns once.
Illustrative example: Shared first three steps don't get duplicated because later branches differ.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Core mechanism
Why replay only correct traces suppressed relative to base?
Preserving Reasoning Diversity in Reinforcement Learning with Verifiable Rewards via Adaptive Replay
Show answer and example
Answer: Verified real base samples store diverse reachable solutions. Gate inactive if unchanged; if suppressed, concentrate on lowered tokens. Detached selection prevents optimization through the gate, and test labels are excluded.
Illustrative example: Restore disappearing algebra steps without repeatedly forcing a still-common diagram solution or inventing archived samples.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Implementation details
May TAR average only active gate trajectories?
Preserving Reasoning Diversity in Reinforcement Learning with Verifiable Rewards via Adaptive Replay
Show answer and example
Answer: No. Divide by all bank traces, including zero gates. Detach gate/token weights and use stable mixed precision. Active-only normalization amplifies rare activation.
Illustrative example: If just one trace is suppressed, active-only averaging makes it stronger than the bank mean.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Method comparison
How does adaptive TAR differ from repeated SFT on fixed archive?
Preserving Reasoning Diversity in Reinforcement Learning with Verifiable Rewards via Adaptive Replay
Show answer and example
Answer: Open the gate only when relative-to-base suppression occurs, focusing suppressed tokens. Fixed replay always supervises, potentially constraining new adaptation.
Illustrative example: Skip a still-common old solution until its likelihood falls.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Basics
Why collect actual correct base samples for TAR memory?
Preserving Reasoning Diversity in Reinforcement Learning with Verifiable Rewards via Adaptive Replay
Show answer and example
Answer: Recover modes base could truly generate and current training suppresses. Generate, verify with training-only checker and choose diverse prototypes. No correct samples means no bank; hand-written gold is not reachability evidence.
Illustrative example: A never-generated base solution cannot be labeled a forgotten skill.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Core mechanism
Why not project teacher/student hidden states into one coordinate system first?
SimOPD: Relational Knowledge Transfer for On-Policy Distillation
Show answer and example
Answer: Compute similarities in each own space, match distributions over paired historical IDs. Freeze teacher/bank and maintain synchronized FIFO. A single historical point cannot provide relational supervision.
Illustrative example: Two teachers can agree a reply is closer to historical A than B without identical coordinates.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Implementation details
When does SimOPD enqueue and why reject a one-item bank?
SimOPD: Relational Knowledge Transfer for On-Policy Distillation
Show answer and example
Answer: After optimization enqueue frozen paired representations captured before it, using synchronized FIFO; first batch only warms up. One bank item makes the distribution constant and uninformative. Only current student representation gets gradients.
Illustrative example: Misaligned FIFO positions compare relations to different answers.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Method comparison
How does relational SimOPD differ from hidden-state MSE?
SimOPD: Relational Knowledge Transfer for On-Policy Distillation
Show answer and example
Answer: MSE requires comparable coordinates. Relationship matching preserves geometry without forcing coordinates and differs from vocabulary distillation: targets are similarities to historical answers.
Illustrative example: Rotating a map changes coordinates without changing city distances.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Basics
Why can SimOPD handle different teacher/student hidden dimensions?
SimOPD: Relational Knowledge Transfer for On-Policy Distillation
Show answer and example
Answer: Each space compares current reply to paired historical replies. Distill distributions indexed by shared answer identities rather than subtract cross-model vectors.
Illustrative example: A 100-dimensional model and wider model can both rank similarities to the same old answers.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Core mechanism
Why train a simple teacher but restart the hierarchical student from base?
MultiAgent-OPD: End-to-End Training of Hierarchical Multi-Agent Systems with On-Policy Distillation
Show answer and example
Answer: Searcher–Answerer learns local task guidance. Restart distinguishes teacher knowledge from hierarchy learning. Shared weights still receive role-specific signals, and Searcher teacher sees real subtask/history, not global gold for all.
Illustrative example: A failed answer can reflect delegation or retrieval; uniform terminal credit misses local causes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Implementation details
Do shared MultiAgent parameters still need role-specific loss masks?
MultiAgent-OPD: End-to-End Training of Hierarchical Multi-Agent Systems with On-Policy Distillation
Show answer and example
Answer: Yes. Roles use distinct advantages and KL. Searcher log ratio isn't causal contribution; mask tool observations. Activity counts environment steps with search, not raw calls.
Illustrative example: Multiple parallel searches in one step should not inflate leader activity without limit.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Method comparison
Why restart phase two from the same base instead of continuing phase one?
MultiAgent-OPD: End-to-End Training of Hierarchical Multi-Agent Systems with On-Policy Distillation
Show answer and example
Answer: Restart isolates teacher learning from hierarchical student improvement. Shared initialization doesn't imply both teacher and student keep updating.
Illustrative example: Build a search teacher first, then train a team from base rather than convert teacher weights directly.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Basics
Why train a simple Searcher–Answerer teacher first?
MultiAgent-OPD: End-to-End Training of Hierarchical Multi-Agent Systems with On-Policy Distillation
Show answer and example
Answer: Team terminal rewards are coarse for search. Train a task-relevant local teacher, then freeze it to guide hierarchical Searcher. Leader and Answerer keep distinct outcome signals.
Illustrative example: Final answer correctness alone can't rate the search summary reliably.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Core mechanism
Why use generation-time cached entropy rather than recalculate after an update?
MGE-PO: Entropy-Aware Credit Assignment for Mathematical Reasoning
Show answer and example
Answer: Credit describes uncertainty at sampling. New policy entropy would mix policy changes into old trajectory judgment. Combine outcome and entropy quadrants with separate curriculum/KL; truncated replies can still shift through recentering.
Illustrative example: A reply becoming more certain after training was not necessarily sampled confidently.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Implementation details
Why can a truncated response have nonzero final advantage after zero direct correction?
MGE-PO: Entropy-Aware Credit Assignment for Mathematical Reasoning
Show answer and example
Answer: Zero direct increment doesn't prevent batch recentering from moving all advantages. Use lagged problem EMA for curriculum and cached sampling-policy entropy, not future generations.
Illustrative example: Positive corrections on peers shift the mean and move an uncorrected reply relatively.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Method comparison
How do MGE-PO quadrants differ from rewarding low entropy alone?
MGE-PO: Entropy-Aware Credit Assignment for Mathematical Reasoning
Show answer and example
Answer: Combine outcome with entropy; confidence is not inherently good. Low-entropy reward alone can reinforce confident errors. Verifiable correctness is needed without leaking evaluation gold.
Illustrative example: Low-entropy correct and low-entropy wrong replies need different credit.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Basics
Are MGE-PO success/failure and high/low entropy the same label?
MGE-PO: Entropy-Aware Credit Assignment for Mathematical Reasoning
Show answer and example
Answer: No. Verified outcome and policy uncertainty are distinct axes; all four combinations occur. Entropy alone is not correctness.
Illustrative example: A model can confidently be wrong or hesitate and get it right.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Core mechanism
Why retain mean pressure after group centering?
GR-OPD: Group-Relative Sequence Credit for On-Policy Distillation
Show answer and example
Answer: Pure centering keeps only within-group ranking and can erase shared correction. Add mean pressure before broadcasting credit to actions. Discount, standardization and clipping alter the practical estimator's guarantees.
Illustrative example: All replies can be similarly unsupported; centering nearly zeroes them while mean term preserves correction.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Implementation details
Does GR-OPD average tokens globally or per response?
GR-OPD: Group-Relative Sequence Credit for On-Policy Distillation
Show answer and example
Answer: Average valid actor tokens within each response then average replies equally. Clip and discount-normalize feedback first. Freeze teacher/old policy/credit. Bare theoretical sums don't imply guarantees under practical clipping and reuse.
Illustrative example: Length alone doesn't give a response more total weight.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Method comparison
How does retaining mean pressure differ from pure group centering?
GR-OPD: Group-Relative Sequence Credit for On-Policy Distillation
Show answer and example
Answer: Pure centering removes teacher's common direction across replies. An extra mean term preserves some shared pressure; its coefficient may be zero. Treat configurations separately.
Illustrative example: If every reply diverges from teacher, relative ranks alone hide shared mismatch.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Basics
Why pool token feedback into sequence scores in GR-OPD?
GR-OPD: Group-Relative Sequence Credit for On-Policy Distillation
Show answer and example
Answer: Local teacher confidence needn't mean good overall reasoning. Discount and aggregate sequence feedback, then compare same-prompt replies for relative credit.
Illustrative example: Teacher preference for formatting tokens doesn't make a wrong trajectory preferable.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Core mechanism
Why does Sink-Sparse's global branch pool sink blocks as well?
Sink-Sparse Hybrid Attention for Efficient Inference in T2I/T2V Diffusion Transformers
Show answer and example
Answer: It supplies low-resolution context over every token; the sparse branch computes fine selected interactions. They combine via their normalizers, not by adding only missing blocks. Excluding sink changes the compensation distribution, and test examples cannot reselect the offline support.
Illustrative example: The sparse branch reads the center closely while the global branch still summarizes the whole image, center included.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Implementation details
Can block mass be calibrated by averaging Q/K before softmax?
Sink-Sparse Hybrid Attention for Efficient Inference in T2I/T2V Diffusion Transformers
Show answer and example
Answer: No. Compute dense token probabilities first, average over query blocks, and sum key blocks to preserve unit row mass. Softmax is nonlinear. Local Python gathers establish correctness, not speed from mask sparsity.
Illustrative example: Two queries can attend opposite ends while their mean attends the center. Early averaging loses the true mass at both ends.
Sections in the Chinese source: 核心公式, 本地复现, 成本判断与边界. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Method comparison
Does a full-support sparse mask alone recover dense attention?
Sink-Sparse Hybrid Attention for Efficient Inference in T2I/T2V Diffusion Transformers
Show answer and example
Answer: Not necessarily. The output still mixes a pooled global compensation branch, whose distribution may differ. Full support with gap 1 can reduce both branches to the same dense computation and is a sufficient degeneration test. General fusion is approximate.
Illustrative example: If the sparse branch sees every token but the global branch averages multiple tokens per block, their mixture can differ from dense alone.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Basics
Is the static sink mask selected online from a new prompt?
Sink-Sparse Hybrid Attention for Efficient Inference in T2I/T2V Diffusion Transformers
Show answer and example
Answer: No. Independent dense calibration aggregates exact block mass, then takes the shortest row prefix satisfying the mass threshold. New inputs reuse the mask. This saves selection cost but depends on cross-prompt and cross-seed stability.
Illustrative example: Fixed calibration may select two regions. A new prompt's high-scoring third region does not silently redefine the mask.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Core mechanism
Why is plane eviction using the next camera path not reading real future video?
Persistent Planar Memory for Video World Models
Show answer and example
Answer: The path can be a provided generation control: it says where the camera will point, not what real pixels will appear. The plane pool prepares for better observations, but the model must generate them. Unknown future observations or gold geometry would be leakage.
Illustrative example: A specified orbit around a table tells where to look without revealing the texture on the back.
Sections in the Chinese source: 背景与主要改动, 核心公式与流程. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Implementation details
Is geometry LoRA enabled at every video denoising step?
Persistent Planar Memory for Video World Models
Show answer and example
Answer: No. Denoise with the frozen stage-one backbone and geometry LoRA off. After generating video, run one extra geometry forward with LoRA to extract planes. Training also uses depth, normals, rasterization, and smoothing, with no rasterization loss in the first thousand steps. Local status is documentation only.
Illustrative example: Generate a 25-frame chunk through multiple steps, then extract planes once and merge the pool.
Sections in the Chinese source: 核心公式与流程, 本地复现与成本判断. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Method comparison
How does a persistent plane pool differ from conditioning only on recent frames?
Persistent Planar Memory for Video World Models
Show answer and example
Answer: Recent context forgets distant views. A world-coordinate pool supplies geometry during revisits and requires occlusion voting, quality deduplication, and eviction. Known future camera pose does not supply real future video.
Illustrative example: A room orbit can reuse an old wall while eviction knows the next camera location, not future pixels.
Sections in the Chinese source: 背景与主要改动, 核心公式与流程. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Basics
Why do long video generators forget scene layout when returning to an old viewpoint?
Persistent Planar Memory for Video World Models
Show answer and example
Answer: A few recent frames omit earlier geometry; storing all pixels or points is costly. PPMem maintains compact rasterizable planes across chunks for revisits. Planes approximate scenes and still need occlusion/error management. Persistence alone is not consistency.
Illustrative example: When a camera returns to a desk, persistent memory can preserve a bookshelf that has been absent from recent frames.
Sections in the Chinese source: 背景与主要改动, 核心公式与流程. The example illustrates the mechanism.
Structured Decisions / Typed Readout: From Generated Sentences to Candidate Distributions
Core mechanism
Why independently score documents rather than put all of them into one Choice?
Jev-as-a-Judge for Brazilian Portuguese Information Retrieval: Relevance Assessment and Reranking
Show answer and example
Answer: Binary prompts estimate whether each document directly answers, making scores sortable across batches after all return. Joint Choice outputs competitive shares whose denominator changes with new documents. Ordinal labeling is separate and retains label order. No BM25 fusion or trained calibration, and zero-relevance queries stay in evaluation.
Illustrative example: Example: Two documents can each be highly likely to answer. In a two-choice distribution they must share probability mass.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Structured Decisions / Typed Readout: From Generated Sentences to Candidate Distributions
Implementation details
Can a missing Portuguese reranker score default to zero?
Jev-as-a-Judge for Brazilian Portuguese Information Retrieval: Relevance Assessment and Reranking
Show answer and example
Answer: No. Collect and match every expected document for the query; reject missing, duplicate or nonfinite entries. Silent zero converts a protocol error into a relevance judgment and changes ranking.
Illustrative example: Example: If the second batch is missing, don't put those documents last and compute metrics anyway.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Structured Decisions / Typed Readout: From Generated Sentences to Candidate Distributions
Method comparison
How does an independent Noul probability differ from adding BM25?
Jev-as-a-Judge for Brazilian Portuguese Information Retrieval: Relevance Assessment and Reranking
Show answer and example
Answer: The reranker uses the independent answer probability directly without mixing BM25. Any fusion is a different method and needs separate evaluation.
Illustrative example: Example: BM25 can retrieve the initial set. Adding it again to the final score changes the specified reranker.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Structured Decisions / Typed Readout: From Generated Sentences to Candidate Distributions
Basics
Are relevance annotation and reranking the same output in Portuguese JEV retrieval?
Jev-as-a-Judge for Brazilian Portuguese Information Retrieval: Relevance Assessment and Reranking
Show answer and example
Answer: No. Annotation assigns ordinal relevance; reranking uses independent probabilities that each document directly answers a query. Metrics and semantics differ; class shares are not independent relevance probabilities.
Illustrative example: Example: Marking one document highly relevant and ordering a collection are distinct output types.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Core mechanism
Why share BMuon budget across MLPs and apply it to input rows and output columns?
BMuon: Budgeting Muon's Spectral Geometry for Structured Compressibility
Show answer and example
Answer: One hidden channel is an input row and output column pair. A shared budget reallocates compressibility across layers and guides both sides' Muon geometry. It shapes training updates, not forward gates or post-hoc independent pruning.
Illustrative example: A 100-channel budget need not split 50/50, but paired edges use the same allocation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Implementation details
Can BMuon reuse the old polar update after an MM budget update?
BMuon: Budgeting Muon's Spectral Geometry for Structured Compressibility
Show answer and example
Answer: No. Use old allocation to compute MM scores, then recompute bilateral polar directions using the new shared allocation and one multiplier. Missing either paired gradient must fail rather than silently decouple.
Illustrative example: A channel moving 0.2→0.8 needs directions based on new μ.
Sections in the Chinese source: 核心公式, 实现与运行. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Method comparison
How does global budgeting differ from equal per-layer retention?
BMuon: Budgeting Muon's Spectral Geometry for Structured Compressibility
Show answer and example
Answer: All blocks compete for total K; fixed ratios preassign capacity per layer. Equal total channels can have different allocations.
Illustrative example: One layer can keep more while another keeps less instead of pruning 50% everywhere.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Basics
What prunable structure is BMuon designed to learn?
BMuon: Budgeting Muon's Spectral Geometry for Structured Compressibility
Show answer and example
Answer: It couples MLP hidden channel input rows and output columns under a common budget. This is optimizer geometry, not extra gates or arbitrary magnitude pruning.
Illustrative example: Consider both edges of a channel together.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Core mechanism
Why validate collective improvement on hidden downstream packages instead of only agent-maintained microbenchmarks?
World of SWECraft: Agent Collaboration at Scale via Simulated Software Worlds
Show answer and example
Answer: The group may overfit visible benchmarks while breaking real dependents. Installing frozen releases in unseen downstream packages tests compatibility and benefits. Failures stay in strict or carry aggregation rather than being dropped, and instruction counts do not measure wall-clock speed.
Illustrative example: Example: A function specialized for self-test inputs can improve a microbenchmark while breaking a downstream parser. Hidden dependencies reveal the regression.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Implementation details
How do strict and carry aggregate speed in correctness-failure rounds?
World of SWECraft: Agent Collaboration at Scale via Simulated Software Worlds
Show answer and example
Answer: Strict assigns failed package tasks speed ratio one; carry uses the most recent passing ratio. Both retain all tasks and take a geometric mean. Instruction-count gains are not wall-clock speedup. Local formulas do not execute the ecosystem or reproduce its 55K sessions.
Illustrative example: Example: If an optimization breaks tests, keep the package in the denominator. Carry uses its last valid speed, not a faster failing run.
Sections in the Chinese source: 核心公式, 本地复现与成本判断. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Method comparison
How does persistent multi-session collaboration differ from one indefinitely long group chat?
World of SWECraft: Agent Collaboration at Scale via Simulated Software Worlds
Show answer and example
Answer: Agents hand off through repositories, issues, PRs, releases, and durable memory, with separate session permissions and budgets. Shared artifacts preserve work without replaying every discussion, but coordination, ownership, and versioning still cost. More agents or larger models do not automatically produce more.
Illustrative example: Example: One maintainer releases a new version and a dependent library adapts using release artifacts, without all chat history being visible every turn.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Basics
Why does World of SWEcraft model a package ecosystem maintained by several agents?
World of SWECraft: Agent Collaboration at Scale via Simulated Software Worlds
Show answer and example
Answer: Fixing one isolated issue does not capture cross-project dependencies or long-term effects. Maintainers continuously change packages, affecting downstream users and requiring coordinated releases alongside local progress. Activity volume does not measure ecosystem quality; verify downstream functionality and resources.
Illustrative example: Example: A package changes a function signature and passes its own tests, but two dependent packages no longer import. Single-issue tests can miss that.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Core mechanism
Why doesn't successful API tape replay prove program equivalence for every input or serve as a candidate acceptance gate?
SCOPE: Selective Certification of Program Edits for Embodied Heuristic Learning
Show answer and example
Answer: Replay checks recorded cases with given state, responses, and exceptions, leaving unseen inputs unknown. It selects divergences worth diagnosing natively rather than requiring rollback of a successful candidate. Only native-environment success updates the accepted program; replay divergence is not environment failure.
Illustrative example: Changed return ordering on an old tape can prompt targeted retesting. A candidate that completed the native task is not rejected solely because an old tape is incompatible.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Implementation details
Is matching only API names and return values enough for stateful SCOPE replay?
SCOPE: Selective Certification of Program Edits for Embodied Heuristic Learning
Show answer and example
Answer: Also match requests and prior state, restore subsequent mutations, exceptions, and aliases, consume the complete tape, and make no extra external calls. Native success updates the accepted program; do not turn replay into acceptance or rollback.
Illustrative example: If two objects share one list, replaying them as independent copies changes later behavior even when the immediate result matches.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Method comparison
Are SCOPE replay divergence and failure in the actual environment the same conclusion?
SCOPE: Selective Certification of Program Edits for Embodied Heuristic Learning
Show answer and example
Answer: No. Divergence means the candidate differs from or cannot use a recorded witness and informs further search. Native failure requires real execution. Unknown is not failure, and replay cannot establish equivalence over all inputs.
Illustrative example: An extra API query can diverge from the tape without showing that the actual task fails.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Basics
Why does SCOPE save stateful API tapes from successful executions?
SCOPE: Selective Certification of Program Edits for Embodied Heuristic Learning
Show answer and example
Answer: Rerunning all earlier scenarios after a program edit is expensive. A full tape records requests, public state before and after, and results or exceptions, enabling same-instance replay and surfacing differences for native diagnosis.
Illustrative example: A complete tape can screen an old successful case without immediately restarting the robot environment.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Core mechanism
Why use the midpoint as gate rather than a separate target?
Waypoint-OPD: Transferring RL-Induced Policy Shifts via Gated On-Policy Distillation
Show answer and example
Answer: Equal endpoint shifts can be consistent or reversed. Compare student-weight-centered early/late directions, then scale endpoint credit only. Freeze gate with low-norm fallback; no requirement to imitate the midpoint.
Illustrative example: Two teachers raise one token eventually, but one first lowers it. Midpoint distinguishes trajectories without teaching that interim state.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Implementation details
Why default the waypoint gate to one for tiny norms?
Waypoint-OPD: Transferring RL-Induced Policy Shifts via Gated On-Policy Distillation
Show answer and example
Answer: Undefined direction is not conflict. Fall back to one, with student-top-k weighted centering and valid-prefix batch scaling. Freeze support, teacher and advantage. Unsampled candidates may receive gradients.
Illustrative example: Near-zero early displacement shouldn't delete all endpoint credit because cosine is unstable.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Method comparison
How does positive cosine consistency differ from endpoint magnitude?
Waypoint-OPD: Transferring RL-Induced Policy Shifts via Gated On-Policy Distillation
Show answer and example
Answer: Magnitude can hide oscillation. Cosine checks alignment of centered early/late shifts. It is evidence about one training path, not correctness or arbitrary cross-run chronology.
Illustrative example: Opposite segments lower gate despite a large final shift.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Basics
Why use a waypoint checkpoint from the same teacher RL run?
Waypoint-OPD: Transferring RL-Induced Policy Shifts via Gated On-Policy Distillation
Show answer and example
Answer: Endpoints hide reversal. Midpoint shows early/late directional consistency and gates endpoint credit without becoming another imitation target.
Illustrative example: Monotonic movement toward B and movement toward A then B can end similarly.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Core mechanism
Why aren't average teacher probabilities and average logits equivalent?
Compatibility Matters: A Framework for Token-Level Teacher Selection in On-Policy Distillation
Show answer and example
Answer: Arithmetic mixes allocate mass across distributions; log averaging and normalization favor shared support, particularly under tail/mode disagreement. Confidence, distance and aligned features are distinct criteria without universally optimal routing.
Illustrative example: Math/code experts can favor different continuations; sharing their mass differs from requiring common support.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Implementation details
Does soft teacher routing mix probabilities or logits, and what if τ=0?
Compatibility Matters: A Framework for Token-Level Teacher Selection in On-Policy Distillation
Show answer and example
Answer: Arithmetic mix probabilities; averaged logits give a geometric target. τ=0 uses hard argmax. Freeze routing/target. Local sampled-token path is not Top-K KL.
Illustrative example: Different teacher modes get different mass under probability and logit mixtures.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Method comparison
What assumptions underlie confidence, Hellinger and hidden-distance routing?
Compatibility Matters: A Framework for Token-Level Teacher Selection in On-Policy Distillation
Show answer and example
Answer: Confidence values certainty, distribution distance values compatibility, and hidden distance requires aligned representations. They measure different properties; unequal dimensions can't be subtracted, and closeness isn't correctness.
Illustrative example: A confident teacher far beyond student skill may receive different weights by rule.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Basics
Why might selecting teachers tokenwise help compared with one per response?
Compatibility Matters: A Framework for Token-Level Teacher Selection in On-Policy Distillation
Show answer and example
Answer: A response mixes expertise and teacher–student compatibility can vary by prefix. Routing is a plausible mechanism, not a universal empirical win.
Illustrative example: Code and math explanations can benefit from different teachers.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Core mechanism
Why use teacher-specific input second moments rather than a shared covariance?
Training-Free On-Policy Distillation
Show answer and example
Answer: They weight output error along actually visited directions. Arbitrary shared statistics change least squares; termination filtering and token normalization matter. Gradient-free still costs generation, rescoring and solving.
Illustrative example: One shared covariance may ignore the domain-specific support of one expert.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Implementation details
Whose responses estimate TF-OPD Grams?
Training-Free On-Policy Distillation
Show answer and example
Answer: Each teacher rescores its own responses after termination/loop filtering; normalize by tokens. Use FP64 linear solves, not explicit inverse, and clean hooks. Random covariance isn't teacher evidence.
Illustrative example: Scoring student-generated text changes teacher visitation statistics.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Method comparison
How does equal-Gram merging differ from summing task vectors?
Training-Free On-Policy Distillation
Show answer and example
Answer: Identical Gram statistics should yield a weighted average, not amplified sum. Orthogonal support can preserve separate functions. Behavior depends on statistics.
Illustrative example: Two identical experts should not produce twice the task update.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Basics
Why does training-free TF-OPD still need teacher-generated data?
Training-Free On-Policy Distillation
Show answer and example
Answer: Closed-form linear merging uses input second moments from each teacher's visited states to determine output matching. No gradients does not mean no generation or calibration.
Illustrative example: Two specialists' statistics reveal different input subspaces.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Core mechanism
Why discard anchor probes and restart, and when do auxiliary rewards apply?
Multi-Reward Hacking: From Aggregation-Induced Drift to Trust-Anchored Defense
Show answer and example
Answer: Separate probes measure reward-induced drift, not supply final weights. Restart from the common baseline to isolate selection. Anchor failure blocks auxiliary credit. Sample under each probe to preserve KL direction.
Illustrative example: Brevity can't raise credit on an incorrect answer if correctness is the anchor.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Implementation details
Why estimate TAG drift on each probe's own trajectories?
Multi-Reward Hacking: From Aggregation-Induced Drift to Trust-Anchored Defense
Show answer and example
Answer: Reverse-KL expectation is under the probe. Baseline sampling changes direction. Pool all generated tokens, not per-response means. A finite-sample estimate may be negative; do not clamp.
Illustrative example: Long and short responses contribute their tokens jointly rather than equal response weights.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Method comparison
How does TAG's low-drift probe selection differ from manually selecting correctness?
Multi-Reward Hacking: From Aggregation-Induced Drift to Trust-Anchored Defense
Show answer and example
Answer: Probe each reward from the same baseline, choose via probe-to-baseline reverse KL, discard probes and retrain. A manually chosen anchor skips that process.
Illustrative example: Fast reward growth can cause large drift, not make it the chosen anchor.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Basics
Why choose a reliable anchor before TAG multi-reward training?
Multi-Reward Hacking: From Aggregation-Induced Drift to Trust-Anchored Defense
Show answer and example
Answer: Auxiliary objectives can be gamed while harming the main goal. Enable auxiliary credit only when the anchor is met; this differs from shrinking its coefficient.
Illustrative example: A short incorrect answer shouldn't get positive brevity credit on its own.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Core mechanism
Why not normalize inverse-probability gradients by current random token count?
Does On-Policy Distillation Need to Complete Every Rollout?
Show answer and example
Answer: Late positions are less observable and inverse probabilities debias the sum. Dividing by a correlated random denominator needn't preserve expectation. Use previous weighted count, detached along with weights. Nonlinear optimizer steps still differ.
Illustrative example: Upweight seldom-observed later steps without claiming one short average equals full generation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Implementation details
How handle partial final blocks and STOPD normalizer?
Does On-Policy Distillation Need to Complete Every Rollout?
Show answer and example
Answer: Use exact coverage probabilities for the short tail and previous batch's weighted count as current normalizer. Keep begin/loss/finish lifecycle against reuse. Do not replace with current random count.
Illustrative example: A maximum length not divisible by block width still has exact tail probabilities.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Method comparison
Does inverse-inclusion weighting make each Adam step identical?
Does On-Policy Distillation Need to Complete Every Rollout?
Show answer and example
Answer: No. The weighted gradient sum can match full-sum expectation, but normalization and nonlinear optimizer state matter. It is not per-batch update or final-training equality.
Illustrative example: Two short draws may average toward the full sum while giving different adaptive steps.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Basics
Why choose STOPD cutoff randomly before generating?
Does On-Policy Distillation Need to Complete Every Rollout?
Show answer and example
Answer: Independent advance sampling defines each position's inclusion probability and saves actual generation. Trimming after generation only saves supervision and can bias selection. Natural EOS remains valid.
Illustrative example: Choose a block cutoff before seeing the answer, not the most favorable length afterward.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Core mechanism
Why compare sorted certainty windows instead of using entropy or high-probability tokens as labels?
Learning to Reason with Selective Internal Feedback
Show answer and example
Answer: Mean confidence can amplify errors. Build a profile using uniform-to-policy KL, then compare eligible same-prompt replies at window and peer levels. This is not entropy or sampled probability; filter ineligible/equal-reward groups and share policy/KL masks.
Illustrative example: Multiple windows distinguish a few extreme positions from broadly uniform certainty.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Implementation details
Should ineligible SIF replies disable KL as well?
Learning to Reason with Selective Internal Feedback
Show answer and example
Answer: Yes. Policy and KL use the same selection mask. Keep original response normalization, eligibility, strict comparisons and sample SD. A tie isn't a win.
Illustrative example: Turning off policy loss but retaining KL would still update a filtered response.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Method comparison
Why isn't SIF's KL(U||π) negative entropy?
Learning to Reason with Selective Internal Feedback
Show answer and example
Answer: It averages log probabilities uniformly; negative entropy weights by policy probabilities. Direction and low-probability sensitivity differ, as does using only sampled tokens.
Illustrative example: A rare token approaching zero affects uniform-to-policy KL differently from policy-weighted entropy.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Basics
Why not treat mean self-certainty as a correct label in SIF?
Learning to Reason with Selective Internal Feedback
Show answer and example
Answer: Confidence may grow for wrong answers. Compare ranked certainty windows across same-prompt replies, then use two-level voting for relative reward. This is not independent truth.
Illustrative example: A confident error does not become right due to a high mean.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Core mechanism
Why retain minimum same-sign peer residual without selecting one whole answer?
Learning from Unverified Peers via Shared Residual Distillation
Show answer and example
Answer: Compare same-question changes to control to isolate local task effects. Conservative common magnitudes limit individual overreach. Student sees only own prefix; agreement is not truth and target cannot reinforce itself as a peer.
Illustrative example: Keep a weakly supported common middle word without declaring an unverified conclusion correct.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Implementation details
Why can an SRD coordinate-capped surrogate be negative?
Learning from Unverified Peers via Shared Residual Distillation
Show answer and example
Answer: It caps signed coordinate contributions, not total KL. Use q0 centering weights, exclude target from its peers and freeze teachers/targets. Clamping negative coordinates changes the objective.
Illustrative example: A negative local coordinate may be valid even when a KL interpretation concerns the full distribution.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Method comparison
How does minmod residual differ from averaging peer logits?
Learning from Unverified Peers via Shared Residual Distillation
Show answer and example
Answer: Keep the minimum magnitude only when all signs agree; otherwise zero. Mean can let strong opinions swamp conflict. Shared agreement can still be biased.
Illustrative example: Two positive peers and one negative don't get a positive residual by simple majority.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Basics
Why not choose an SRD answer by majority vote?
Learning from Unverified Peers via Shared Residual Distillation
Show answer and example
Answer: Unverified peers can all be wrong yet contain useful local signals. Compare same-question peers with cross-question controls; retain minimum common signed residual instead of declaring a whole answer correct.
Illustrative example: Failed traces can still share one reasonable intermediate step.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Core mechanism
Why use discounted potential differences and not treat timeout as termination?
Progress Reward Model for Reinforcement Learning via Large Language Models
Show answer and example
Answer: Neighbor terms telescope to endpoints. Correct absorbing boundary prevents terminal potential changing the objective. A timeout may leave future progress possible, so don't zero it automatically. Subtask encoding and shaping remain independent.
Illustrative example: Zeroing potential when time expires near a goal can fabricate a progress penalty.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Implementation details
Why distinguish true termination from time truncation in PRM4RL?
Progress Reward Model for Reinforcement Learning via Large Language Models
Show answer and example
Answer: An absorbing terminal can have zero potential; time-limited states can retain successor value. Use the formula absent explicit termination and review generated functions.
Illustrative example: Timeout with a resumable task differs from actual success or failure.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Method comparison
How does potential shaping differ from rewarding raw progress at each step?
Progress Reward Model for Reinforcement Learning via Large Language Models
Show answer and example
Answer: Discounted differences compare transitions and telescope, while repeated high potential can reward idling. Terminal conditions matter; differencing alone does not guarantee policy invariance.
Illustrative example: Staying at a high-progress state without finishing can accrue reward under raw per-step scores.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Basics
Why does PRM4RL provide subtask semantics and a progress potential?
Progress Reward Model for Reinforcement Learning via Large Language Models
Show answer and example
Answer: Subtask meaning enriches state with current phase; potential differences supply dense progress despite sparse terminal success. One changes inputs, the other rewards, and planning can be offline.
Illustrative example: A robot knows it is grasping and receives progress for moving closer.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Core mechanism
Why expire OPRD local reminders after the relevant next turn?
OPRD: On-Policy Reminder Distillation for Long-Horizon Coding Agents
Show answer and example
Answer: A specific observation triggers a short-lived constraint. Global/local scopes differ; teacher rescores actual prefixes and deployment receives no reminder. Normalize success imitation and failure distillation separately.
Illustrative example: A dependency reminder from compilation should expire after the dependency is fixed, not remain pending forever.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Implementation details
How choose reminder-OPRD Top-100 and refresh teacher?
OPRD: On-Policy Reminder Distillation for Long-Horizon Coding Agents
Show answer and example
Answer: Rank shared support by elementwise max of both logp vectors, then normalize each on it. Refresh teacher at specified milestones, otherwise freeze. Average each trace then success/failure separately.
Illustrative example: Independent supports compare different events in coordinatewise KL.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Method comparison
How do global and post-observation local reminder scopes differ?
OPRD: On-Policy Reminder Distillation for Long-Horizon Coding Agents
Show answer and example
Answer: Global reminders persist; local ones apply only to the next turn after that observation and then expire. Permanent local reminders change unrelated conditions.
Illustrative example: A fix for one error belongs to the next attempt, not every later turn.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Basics
Why sample reminder-OPRD students without reminders?
OPRD: On-Policy Reminder Distillation for Long-Horizon Coding Agents
Show answer and example
Answer: Deployment is unhinted. After failure a privileged teacher rescores actual prefixes using reminders to internalize constraints. Successful traces use self-imitation rather than teacher rewrites.
Illustrative example: A missed testing constraint can guide training scoring while inference sees only original task.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Core mechanism
Why match visited set, remaining budget and phase when comparing operations?
OpCredit: Operation-Aware Policy Learning for Agentic Knowledge Graph Question Answering
Show answer and example
Answer: The same entity can have different future opportunities. BOC matches context and deduplicates by trajectory; FTC uses training answer neighborhoods for local improvement. Keep answers out of actor input and observations out of policy loss.
Illustrative example: Two visits to one node differ if only one has explored its neighbors.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Implementation details
Why not match OpCredit branches by active entity alone?
OpCredit: Operation-Aware Policy Learning for Agentic Knowledge Graph Question Answering
Show answer and example
Answer: Match problem, visited set, budget and phase to compare similar opportunities. Count each trajectory once. FTC uses the frontier's best chance, not graph-wide mean distance. Detach credit and reference.
Illustrative example: Arriving with no budget differs from arriving with resources to search.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Method comparison
What does BOC branch comparison use versus FTC frontier progress?
OpCredit: Operation-Aware Policy Learning for Agentic Knowledge Graph Question Answering
Show answer and example
Answer: BOC reuses outcomes from matched states; FTC assesses local frontier against a training-only answer neighborhood. Training labels can score credit afterward but remain hidden during sampling. They are distinct signals.
Illustrative example: Closeness to the training answer may score a neighbor after rollout, never enter the search request.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Basics
Why assign OpCredit search credit to individual operations?
OpCredit: Operation-Aware Policy Learning for Agentic Knowledge Graph Question Answering
Show answer and example
Answer: Successful trajectories can detour and failed ones can explore usefully. Branch outcomes and frontier changes score local operations, then align generated reasoning/action tokens, not environment observations.
Illustrative example: Finding a key entity can be useful even if the final answer fails.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Core mechanism
Why compare vocabulary-wide JS after removing each reward block?
Show answer and example
Answer: A sampled token may miss movement among other candidates. Full-distribution changes associate reward information with positions. Mean-preserving weights apply to both signs with bounded magnitude. Verifier context belongs only in training replay.
Illustrative example: Removing SQL-format feedback may shift mass between comma and parenthesis while the sampled token is unchanged.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Implementation details
Why use uniform ones when all MRTPO relevance is zero?
Show answer and example
Answer: If localization is unavailable, uniform weights retain total credit. Keep response mean one before second-moment limits and batch normalization. Zeros delete the objective.
Illustrative example: An empty format block should not erase the format advantage.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Method comparison
How does information ablation differ from sampled-token probability changes?
Show answer and example
Answer: Use full-vocabulary JS under the same frozen policy. A single token can miss mass redistribution. Association is not causal correctness, and reward info cannot enter original student sampling.
Illustrative example: The sampled token may stay constant while alternative probabilities move.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Basics
Why not broadcast every MRTPO reward equally over a response?
Show answer and example
Answer: Format, executability and correctness can depend on different tokens. Removing one information block and measuring distribution shift identifies likely relevant positions.
Illustrative example: SQL grammar and numerical correctness may point to different tokens.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Core mechanism
Why group by pre-action knowledge rather than post-action knowledge?
KnowGPO: Knowledge-Grouped Policy Optimization for Search Agents
Show answer and example
Answer: Use shared initial facts to compare action benefits without conditioning on gains. A training verifier monotonically accumulates facts; combine process/terminal advantages for generated tokens only, hiding checklist from the actor.
Illustrative example: Grouping a successful and unsuccessful search by resulting facts separates the two outcomes before comparison.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Implementation details
How handle KnowGPO answer steps and tool observations in loss?
KnowGPO: Knowledge-Grouped Policy Optimization for Search Agents
Show answer and example
Answer: Answer steps skip process grouping but keep the terminal mixture term. Advantage covers generated reasoning/actions only. Keep verifier facts and tool observations out of actor targets. Singleton process advantage is zero.
Illustrative example: Environment search results are not model-generated tokens.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Method comparison
Why not group KnowGPO by state after the action?
KnowGPO: Knowledge-Grouped Policy Optimization for Search Agents
Show answer and example
Answer: Credit compares actions from a common starting point. Grouping by outcome separates actions that add different facts. Follow algorithmic pre-state despite prose ambiguity.
Illustrative example: An action finding a new fact should be compared to a miss from the same known facts.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Basics
Why group KnowGPO steps by known fact sets?
KnowGPO: Knowledge-Grouped Policy Optimization for Search Agents
Show answer and example
Answer: Different queries/pages can yield the same knowledge. Verified atomic conditions build a monotone state, enabling comparisons of what actions add from a common start.
Illustrative example: Different wording can establish the same person and year and share a pre-state.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Core mechanism
Why index EliteReplay by realized approach and balance problem before cell?
EliteReplay: Preserving Diverse Solutions in RL Post-Training via Quality-Diversity Search
Show answer and example
Answer: Requested methods can differ from generated ones. Use actual descriptor and retain shortest verified answer in each cell. Problem-first balancing prevents archive size from allocating weight. This isn't IID success caching.
Illustrative example: A requested geometry solution using algebra belongs in algebra, or identical methods masquerade as diversity.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Implementation details
Why balance problems before occupied cells?
EliteReplay: Preserving Diverse Solutions in RL Post-Training via Quality-Diversity Search
Show answer and example
Answer: Otherwise larger archives dominate replay. Average response tokens, mix archive and GRPO as specified, clear padding before exponentiating and use no test gold for archiving.
Illustrative example: More occupied cells should not give one problem ten times the sampling weight.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Method comparison
How does shortest-per-cell MAP-Elites differ from a global shortest answer?
EliteReplay: Preserving Diverse Solutions in RL Post-Training via Quality-Diversity Search
Show answer and example
Answer: Cell-local replacement preserves other strategies. Store actual descriptors, not requested approaches, which may not match generated behavior.
Illustrative example: A proof requested as induction but generated by enumeration belongs to the enumeration cell.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Basics
Why maintain a quality–diversity archive per EliteReplay problem?
EliteReplay: Preserving Diverse Solutions in RL Post-Training via Quality-Diversity Search
Show answer and example
Answer: Accuracy alone can shrink solution coverage. Keep the shortest verified solution per approach/structure cell to preserve distinct successful strategies rather than store ordinary IID replay.
Illustrative example: Separate algebra and geometry cells so the common strategy doesn't overwrite the other.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Core mechanism
Why preserve generated order when assigning DIVERGE diversity credit?
Training Reasoning Models to Generate Diverse Scientific Ideas
Show answer and example
Answer: Each idea sees preceding ideas, so nearest predecessor reflects its actual choice. Reordering changes credit. Evaluation's discounted reordered metric is separate; semantic distance and quality need independent judging.
Illustrative example: A second paraphrase deserves lower novelty under the original order; rearranging does not rewrite its training reward.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Implementation details
Which fields count toward DIVERGE length penalties?
Training Reasoning Models to Generate Diverse Scientific Ideas
Show answer and example
Answer: Count tokens in problem_gap, core_claim and method, not inspiration or think, using the intended tokenizer. Schema failure gives zero without a judge call. Character count isn't a substitute.
Illustrative example: A long inspiration citation isn't part of the specified length term.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Method comparison
Can DIVERGE training set reward and evaluated Creative Utility be exchanged?
Training Reasoning Models to Generate Diverse Scientific Ideas
Show answer and example
Answer: Training uses distance to the closest predecessor in generation order; evaluation can reorder, discount and normalize. Greedy evaluation is only a lower bound on maximum CU.
Illustrative example: Moving the best idea first can alter evaluation but doesn't change original training order.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Basics
Why generate a whole idea set instead of sampling independently and deduplicating?
Training Reasoning Models to Generate Diverse Scientific Ideas
Show answer and example
Answer: Later ideas see predecessors and can seek genuinely different directions. Reward combines quality and dissimilarity. Different strings do not prove scientific novelty; semantic judging matters.
Illustrative example: A second approach can cover a gap rather than paraphrase the first.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Core mechanism
Why match semantics as well as recovered preference, and may negative risk be clipped?
CleanRM: Reward Modeling from Noisy Feedback with a Few Clean Feedback for RLHF
Show answer and example
Answer: Matching preference proportions alone can align semantically unrelated examples. Joint optimal transport uses representation distances and recovered preferences to estimate both noise rates. Fixed corrected loss can be negative; clipping changes its expectation, and inaccurate rates also break unbiasedness.
Illustrative example: Two domains can have half positive labels while one flips them often; use semantic representations to expose the difference.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Implementation details
Should CleanRM clamp recovered labels or corrected losses to their conventional ranges?
CleanRM: Reward Modeling from Noisy Feedback with a Few Clean Feedback for RLHF
Show answer and example
Answer: No. Recovered preferences can fall outside [0,1] and corrected risks are signed. Clamping changes the debiasing objective. Require rates summing to less than one and freeze the estimator during RM updates.
Illustrative example: A valid negative corrected loss should not be reset to zero for prettier curves.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Method comparison
How does freezing the CleanRM noise estimator differ from jointly updating it with the reward model?
CleanRM: Reward Modeling from Noisy Feedback with a Few Clean Feedback for RLHF
Show answer and example
Answer: A frozen estimator defines a fixed corrected risk. Joint updates move its coefficients and can lower loss by changing the explanation of noise. This is a different objective without the same fixed-rate risk interpretation.
Illustrative example: Estimate label flips before training; don't reinterpret temporarily misclassified samples as noisier.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Basics
Why distinguish two preference-flip rates rather than estimate one overall error rate?
CleanRM: Reward Modeling from Noisy Feedback with a Few Clean Feedback for RLHF
Show answer and example
Answer: True positives flipped negative and true negatives flipped positive can have different causes. Estimating each conditional rate corrects observed-label risks separately; one error rate hides asymmetry.
Illustrative example: A length-biased annotator may reject correct short answers at a different rate than they accept incorrect long answers.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Core mechanism
Why match student prefixes across views and keep full student normalization?
Condition-Contrastive Extrapolation for On-Policy Distillation of Language Models
Show answer and example
Answer: Fixed prefixes isolate premise removal. Teacher auxiliary target is normalized on candidate support but student uses full-vocabulary softmax, retaining outside gradients. Gate controls only auxiliary correction; base OPD stays.
Illustrative example: If premise and student reasoning both change, likelihood differences can't isolate the premise.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Implementation details
Where is CCE entropy computed, and is student renormalized on support?
Condition-Contrastive Extrapolation for On-Policy Distillation of Language Models
Show answer and example
Answer: Compute entropy on full teacher vocabulary before top-set union. Normalize auxiliary target within support but preserve student full softmax for outside-support gradients. Denominator counts all valid positions.
Illustrative example: Student support renormalization would hide outside mass and change loss.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Method comparison
Does disabling CCE auxiliary gating remove all OPD at a position?
Condition-Contrastive Extrapolation for On-Policy Distillation of Language Models
Show answer and example
Answer: No. Sampled PPO-OPD remains; only auxiliary forward KL is removed. Premise eligibility and entropy gate the extra term.
Illustrative example: A low-entropy format token still receives base distillation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Basics
Why mask a valid premise in CCE's teacher contrast?
Condition-Contrastive Extrapolation for On-Policy Distillation of Language Models
Show answer and example
Answer: A teacher may follow a mistaken student prefix instead of using conditions. Matched ablation reveals condition-sensitive directions for an extrapolated auxiliary target. Student always sees the full problem.
Illustrative example: Removing a numerical premise can identify which next tokens depend on it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Core mechanism
Why is SoftCache's step-size control more than refresh versus reuse?
SoftCache: Adaptive Step Size Caching for Diffusion Transformers
Show answer and example
Answer: Standard caches choose whether to recompute on a fixed schedule. SoftCache uses remaining staleness headroom to adjust the step and rebuild the remaining grid, so skipped nodes are not revisited. Extrapolation needs fresh anchors and early or fast-changing regions restrict jumps.
Illustrative example: Skip two of ten planned denoising nodes only if reliable, then reschedule so the same nodes are not computed later.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Implementation details
Can two-point extrapolation run with one fresh residual anchor?
SoftCache: Adaptive Step Size Caching for Diffusion Transformers
Show answer and example
Answer: No; one anchor gives a constant, and two anchors must be at distinct times. For t<0.1 or probe change above 0.18, do not jump. Clip step size to the endpoint and use a separate cache for each sample.
Illustrative example: The first residual has no prior timestamp to form a slope; dividing by a nonexistent time difference is invalid.
Sections in the Chinese source: 核心公式, 本地复现, 代码映射与边界. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Method comparison
How do relative probe changes and residual extrapolation serve different purposes?
SoftCache: Adaptive Step Size Caching for Diffusion Transformers
Show answer and example
Answer: Accumulated changes estimate staleness headroom and decide refresh/step size. Two fresh residuals estimate a trend for the reused value. They are scheduling versus approximation signals and not interchangeable. One anchor cannot define a slope.
Illustrative example: The probe can say change is small without giving the current residual; the recent actual residuals estimate it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Basics
Why study which diffusion forwards to skip rather than only shrink the model?
SoftCache: Adaptive Step Size Caching for Diffusion Transformers
Show answer and example
Answer: Sampling calls the same network repeatedly; nearby states may share computation, but unconditional skipping can compound error. SoftCache adapts reuse and progress using change signals while retaining the model. Cache reliability is not a quality metric.
Illustrative example: Reuse when nearby denoising states change little and recompute when they move quickly.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Core mechanism
Why can a recurrent block save stored layers without reducing executed depth?
Retrofitting Recurrent Vision-Language Models from Pretrained Backbones
Show answer and example
Answer: Repetition stores one set of weights but pays for every application. Pruning can damage language and the visual interface, so restore text, align the projector, then restore instruction behavior. Stored parameters and inference steps are different axes.
Illustrative example: Five core layers repeated three times execute 15 layers' worth of work while storing five sets of weights.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Implementation details
What is the local status of frozen teacher and inter-loop KV sharing?
Retrofitting Recurrent Vision-Language Models from Pretrained Backbones
Show answer and example
Answer: The teacher is a separate, intact frozen copy with detached outputs; the student updates only stage-allowed parameters. Local checks verify shared calls and manual unrolling, not incremental decoding with inter-loop KV reuse or throughput.
Illustrative example: Three calls to the same core confirm parameter sharing, not that all three share one KV cache correctly.
Sections in the Chinese source: 核心公式, 本地复现, 代码映射与边界. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Method comparison
Why align the projector separately during staged recovery after pruning?
Retrofitting Recurrent Vision-Language Models from Pretrained Backbones
Show answer and example
Answer: Surgery can disrupt both language and vision–language alignment. Recover text first, align only the projector next, then restore instructions. The projector stage excludes KD, unlike the two surrounding recovery phases.
Illustrative example: Restore text continuation before adapting image features to avoid confounding language damage and projector mismatch.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Basics
What do prelude, recurrent core, and coda do in RR-VLM?
Retrofitting Recurrent Vision-Language Models from Pretrained Backbones
Show answer and example
Answer: The prelude computes context h. The core combines h with the previous state and updates s repeatedly. The coda reads the final state. They are stages, not independent voting models. Do not overwrite fixed context as though it were recurrent state.
Illustrative example: Build h once, update s₁ then s₂, and let the coda read s₂.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Core mechanism
Why rerotate K but not V when compacting sparse KV positions?
Compression Does Not End at Selection: Post-Selection Positional Geometry for Sparse KV Caches
Show answer and example
Answer: Undo each key's old rotation then apply its compact position; values reorder without rotation. Align unequal heads to a common right anchor and shift queries/generation. This changes coordinates, not evidence selection; scaled inverse is not simply transpose.
Illustrative example: Move old positions ten and 100 to adjacent slots, rerotate K, reorder V, and start questions at the new anchor.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Implementation details
Why use a common M rather than per-head continuation positions?
Compression Does Not End at Selection: Post-Selection Positional Geometry for Sparse KV Caches
Show answer and example
Answer: M is the maximum kept length across layers/heads. Right-align all heads, start questions at M and answer input at M+Q. Different anchors would disagree. Invert scaled RoPE correctly.
Illustrative example: Heads with eight and four entries both start questions at eight; right-align the four retained entries.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Method comparison
How does position reconstruction differ from prefilling selected text again?
Compression Does Not End at Selection: Post-Selection Positional Geometry for Sparse KV Caches
Show answer and example
Answer: Invert and reapply key rotations, and reorder V, without recomputing backbone representations. Re-prefill recalculates KV from text and pays a different cost.
Illustrative example: Transform existing K rather than concatenating three text spans and processing from layer one.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Basics
What does RoPERelay fix after selecting sparse KV?
Compression Does Not End at Selection: Post-Selection Positional Geometry for Sparse KV Caches
Show answer and example
Answer: Retained keys still carry old position phases. Sort temporally, compact coordinates, and align subsequent question and generation without reselecting contents.
Illustrative example: Retained positions 0, 100, and 500 require new coordinates and a defined question start.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Core mechanism
Why scale reader outputs but writer inputs in RoleQ?
RoleQ: Role-Aligned Scales for Converting Language Models to Ternary
Show answer and example
Answer: Reader output units extract information; writer inputs contribute into residual space. A uniform row axis would mix functional units. Exact unweighted ternary vector solutions are not global model optima; full flow includes rotation, compensation, and distillation.
Illustrative example: One output-projection column spreads one input feature across residual dimensions and needs its own functional scale.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Implementation details
Why pack and unpack ternary values and count scales?
RoleQ: Role-Aligned Scales for Converting Language Models to Ternary
Show answer and example
Answer: Theoretical trit width is not storage. Pack five trits per byte and count scales and padding; all-zero vectors use zero scale/codes. Nonzero count alone is not byte evidence.
Illustrative example: Count tail padding and per-unit scales for 77 weights instead of multiplying by log₂3.
Sections in the Chinese source: 核心公式, 结果与独立对照. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Method comparison
How does exact ternary vector search differ from a fixed threshold?
RoleQ: Role-Aligned Scales for Converting Language Models to Ternary
Show answer and example
Answer: Search every nonempty absolute-magnitude sorted prefix for optimal support and scale under the unweighted vector objective. A threshold tries one support; local exactness is not global model optimality.
Illustrative example: A 0.7×mean threshold can miss a support included in full prefix search.
Sections in the Chinese source: 核心公式, 结果与独立对照. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Basics
Why assign scales by role instead of one stored axis?
RoleQ: Role-Aligned Scales for Converting Language Models to Ternary
Show answer and example
Answer: Reading and writing map different functional units to rows or columns. Scale those units independently rather than combining writer contributions.
Illustrative example: MLP input outputs and MLP output inputs represent corresponding hidden channels.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Core mechanism
Why can a remote KV hit be slower than re-prefill?
When the KV Cache Is Slower Than the GPU: Scaling Remote KV Tiers for Context-Parallel Prefill
Show answer and example
Answer: Fetch has fixed request latency and volume-dependent transfer; recompute depends on prefill throughput. Break-even depends on length and bandwidth, and asynchronous shards must avoid head-of-line blocking. Hit rate is not latency. No local remote implementation or GPU results.
Illustrative example: A short hit can spend more time opening the connection than local recompute takes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Implementation details
Why use per-PCP-rank remote instances and async loading?
When the KV Cache Is Slower Than the GPU: Scaling Remote KV Tiers for Context-Parallel Prefill
Show answer and example
Answer: MLA cache is sharded by rank; independent instances and async loads reduce head-of-line blocking. Validation needs the multi-node service and workload, not a single A100/A30 fixture.
Illustrative example: A large request on one rank should not hold a small load on another.
Sections in the Chinese source: 背景与主要改动, 成本判断. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Method comparison
Does B>sR with fixed T0 guarantee fetch is faster at every length?
When the KV Cache Is Slower Than the GPU: Scaling Remote KV Tiers for Context-Parallel Prefill
Show answer and example
Answer: No. It gives asymptotic bandwidth advantage. Compare T0+sn/B against n/R for finite requests; fixed cost can dominate short ones.
Illustrative example: Fast per-token transfer does not erase first-packet latency.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Basics
Why analyze fetch versus re-prefill break-even first?
When the KV Cache Is Slower Than the GPU: Scaling Remote KV Tiers for Context-Parallel Prefill
Show answer and example
Answer: Remote hits pay request and transfer costs; GPU recompute can be fast. Compare both paths by length and bandwidth, not only hit rate.
Illustrative example: Fetching a short cached prefix can be slower than recreating KV.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Structured Decisions / Candidate Invariance: Order, Competitors, and Calibration
Core mechanism
Why combine isolated attention and equal starting positions in Prism-1?
Prism-1: Position- and Rival-Independent Scoring for System One Models
Show answer and example
Answer: The score may depend only on shared state, question and its own option. Isolation prevents cross-reading; equal positions remove order effects. Joint softmax may change the denominator but should preserve old odds. This does not suit cross-option choices such as 'none of the above' or prevent injection in shared state.
Illustrative example: Example: A new tool can win and lower old shares but insertion position alone should not swap the old tools' ranking.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Structured Decisions / Candidate Invariance: Order, Competitors, and Calibration
Implementation details
Can Prism-1 option blocks read each other or start at different positions?
Prism-1: Position- and Rival-Independent Scoring for System One Models
Show answer and example
Answer: No. Each reads shared context and attends causally within its own block; all start at the same position. Otherwise independent scoring breaks. This isolation does not block injection in shared context.
Illustrative example: Example: Inserting a longer option must not shift later options' position IDs.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Structured Decisions / Candidate Invariance: Order, Competitors, and Calibration
Method comparison
Why doesn't Prism-1's option independence suit 'none of the above'?
Prism-1: Position- and Rival-Independent Scoring for System One Models
Show answer and example
Answer: Independent scores see shared state, question and the option itself. 'None of the above' depends on the validity of other options and needs interaction, so imposing the same Luce constraint is inappropriate. This is a structural boundary, not a routine training error.
Illustrative example: Example: Adding a correct answer should invalidate 'none of the above', which an isolated option cannot determine.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Structured Decisions / Candidate Invariance: Order, Competitors, and Calibration
Basics
Must Prism-1's original probabilities stay numerically unchanged after a third option?
Prism-1: Position- and Rival-Independent Scoring for System One Models
Show answer and example
Answer: No. The denominator can increase and both shares can fall. Preserve their relative odds and order; the new option may win. The consistency constraint does not require ignoring additions.
Illustrative example: Example: Original 2:1 odds can become probabilities 0.4 and 0.2 after adding C, still 2:1.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Core mechanism
Why use matching length offline but a two-byte lookup online?
Patch where the text is new: a compression view of where byte-level models should spend compute
Show answer and example
Answer: Longer recent matches define offline novelty labels. Deployment uses a prefix-indexed table and low-count fallback instead of reading a future span. Offline labels are not available when deciding a boundary.
Illustrative example: Train with full template matches, then estimate from the two already generated bytes online.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Implementation details
How does Novelty Patching handle rare byte pairs?
Patch where the text is new: a compression view of where byte-level models should spend compute
Show answer and example
Answer: Below 20 observations, back off to one-byte means, then the global mean. Calibrate τ on training bytes and reset history at document boundaries.
Illustrative example: One observed match should not determine online boundaries for a rare pair.
Sections in the Chinese source: 核心公式, 数据协议. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Method comparison
How does a two-byte table differ from recomputing future match length online?
Patch where the text is new: a compression view of where byte-level models should spend compute
Show answer and example
Answer: Future match lengths are offline labels. Deployment uses already generated bytes and a frozen threshold; looking past the current position leaks future information.
Illustrative example: Do not inspect whether the next 16 unseen bytes occurred before generating them.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Basics
Why use recent repetition to choose byte patches?
Patch where the text is new: a compression view of where byte-level models should spend compute
Show answer and example
Answer: Byte patches reduce global calls. Recent match length captures repeatability, and a cheap two-byte predictor estimates novelty instead of relying only on entropy.
Illustrative example: A recently seen span may be reusable, whereas a novel span may need finer boundaries.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Core mechanism
Why doesn't lower Muon loss at equal steps prove superiority at equal compute or tuning cost?
When Does Muon Actually Win? A Paired Data-, Time-, and Tuning-Budget Benchmark
Show answer and example
Answer: Step latency, throughput, and tuning can differ. Pair initialization, data order, and candidates; compare processed data, elapsed training time, and fixed tuning budget separately. Freeze selection before test.
Illustrative example: Twice-slower steps can win at 100 steps but lose at ten minutes; retuning for time confounds the comparison.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Implementation details
Why sample the same seed indices in paired bootstrap?
When Does Muon Actually Win? A Paired Data-, Time-, and Tuning-Budget Benchmark
Show answer and example
Answer: Paired trajectories share initialization and data order. Mismatching seeds loses correlation. Integrate AUC over common measurement support and freeze validation-selected settings for all budget axes.
Illustrative example: Sample seed 42 for both optimizers, not 42 for one and 44 for the other.
Sections in the Chinese source: 核心公式, 结果与独立对照. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Method comparison
Why isn't random-access regret on a fixed candidate pool prospective adaptive HPO?
When Does Muon Actually Win? A Paired Data-, Time-, and Tuning-Budget Benchmark
Show answer and example
Answer: It studies random order within precomputed candidates. Adaptive HPO uses prior results to propose or choose new trials, with different information and time costs.
Illustrative example: Randomly inspect 23 existing settings versus inventing a 24th from feedback.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Basics
Why report data, elapsed time, and tuning costs separately?
When Does Muon Actually Win? A Paired Data-, Time-, and Tuning-Budget Benchmark
Show answer and example
Answer: Per-step and per-second quality and the cost of finding parameters differ. Equal-data gains need not hold at equal time or search budget.
Illustrative example: Lower loss for equal tokens can still lose after one hour if steps are slower.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Structured Decisions / Typed Readout: From Generated Sentences to Candidate Distributions
Core mechanism
Why read options using final attention Q/K rather than a separate classifier head?
MJev-Spatial: System One Spatial Intelligence via Native Attention Matching
Show answer and example
Answer: Use the decision marker as query and candidate end markers as keys in the backbone's own projections. This shares a readout for options, numeric buckets and regions. Position encoding and grouped-query mapping matter. Points still need staged selection and crops from the original image; classification alone does not ensure spatial precision.
Illustrative example: Example: Select a coarse region, then crop it from the original image. Repeatedly cropping the last resized image changes coordinates and detail.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Structured Decisions / Typed Readout: From Generated Sentences to Candidate Distributions
Implementation details
Why average multistage spatial losses within each original question first?
MJev-Spatial: System One Spatial Intelligence via Native Attention Matching
Show answer and example
Answer: Otherwise questions with more refinement stages receive more weight. Average stage cross-entropies per question, then average questions.
Illustrative example: Example: Three localization choices should not automatically count three times as much as one choice.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Structured Decisions / Typed Readout: From Generated Sentences to Candidate Distributions
Method comparison
How does two-stage counting differ from one class per integer?
MJev-Spatial: System One Spatial Intelligence via Native Attention Matching
Show answer and example
Answer: Two smaller candidate sets compose the result as 32a₁+a₂, covering 0–1023. A large head directly scores each integer. Reusing finite-choice readout is useful, but first-stage errors affect the result; matching output spaces do not imply equivalent training.
Illustrative example: Example: a₁=2 and a₂=3 represent 67, not the sum of two independent counts.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Structured Decisions / Typed Readout: From Generated Sentences to Candidate Distributions
Basics
How does MJev-Spatial express a 2D point as classification?
MJev-Spatial: System One Spatial Intelligence via Native Attention Matching
Show answer and example
Answer: Treat image regions as candidates and select progressively finer regions. Quantities such as counts also use buckets or staged readout. The output is typed choice, not free-form coordinate parsing.
Illustrative example: Example: Pick the upper-right region, then one cell in its next 3×3 grid.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Core mechanism
Should MirrorMoE use mixed mirror probabilities to weight outputs?
Rethinking Routing Exploration in Modern Mixture-of-Experts
Show answer and example
Answer: No. Mix forward and negated-logit distributions for exploratory selection only, annealing back to standard routing. Selected outputs retain original probabilities. Track actual dispatch separately; mixing or renormalizing output weights changes the mechanism.
Illustrative example: A low-probability expert can run for exposure without receiving a larger output coefficient.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Implementation details
Which probabilities and frequencies enter MirrorMoE's load term?
Rethinking Routing Exploration in Modern Mixture-of-Experts
Show answer and example
Answer: Use actual dispatch frequencies and original ξ, not mixed p. Aggregate outputs with original weights and Z=1 without renormalizing selected experts. Selection, load, and aggregation differ.
Illustrative example: Mirror probability can change whether an expert runs, but not its original output weight.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Method comparison
How does mirror selection differ from arbitrary router noise?
Rethinking Routing Exploration in Modern Mixture-of-Experts
Show answer and example
Answer: Apply softmax to negated logits and mix with original probabilities using an annealing coefficient. Arbitrary noise does not reproduce its distribution or endpoint.
Illustrative example: At ρ=0, selection returns to ordinary top-k.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Basics
Why select near-boundary experts early?
Rethinking Routing Exploration in Modern Mixture-of-Experts
Show answer and example
Answer: Dominant shared experts and many fine-grained combinations can obscure alternatives. Mirror routing temporarily suppresses some usual winners to train other combinations, then anneals back.
Illustrative example: Avoid always giving the same winners every early token.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Core mechanism
Why retain rare exponents and overflow rather than drop them in LCT?
LCT: Losslessly Compressed Tensors with GPU friendly execution for Training
Show answer and example
Answer: Bit-exact recovery requires escape codes with raw rare exponents and raw overflow after stream exhaustion. Sign/fraction remain separate and a fixed codebook avoids relearning. This is lossless compression, not quantization.
Illustrative example: A rare exponent uses escape plus eight original bits; overflow stays raw rather than becoming zero.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Implementation details
Why compare raw bits in LCT round trips?
LCT: Losslessly Compressed Tensors with GPU friendly execution for Training
Show answer and example
Answer: Preserve signed zeros, subnormals, infinities, and NaN payloads. Numeric equality can miss these. Test all exponents, centers, and overflow paths.
Illustrative example: +0 and −0 compare equal numerically but have different bits.
Sections in the Chinese source: 核心公式, 数据协议与运行. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Method comparison
How does bounded exponent coding differ from zlib on a tensor?
LCT: Losslessly Compressed Tensors with GPU friendly execution for Training
Show answer and example
Answer: LCT specifies a fixed table, length cap, escapes, stream budget, and raw overflow tail. General compression does not provide that layout. Count codebooks and metadata in total bytes.
Illustrative example: Smaller payload may still produce larger total output after metadata.
Sections in the Chinese source: 背景与主要改动, 结果与独立对照. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Basics
Why store exponents separately from sign and mantissa?
LCT: Losslessly Compressed Tensors with GPU friendly execution for Training
Show answer and example
Answer: Exponents are often concentrated and compressible. Preserve sign/fraction verbatim, use a fixed bounded code, and handle runtime center shifts without losing bits.
Illustrative example: Shared exponent patterns can compress while fraction bits remain intact.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why detach only the explicit codebook shortcut, not the entire noisy input?
Designing Transformers for Continuous Diffusion Language Models
Show answer and example
Answer: Both paths share a codebook. Detaching shortcut gradient still lets the codebook learn through noisy input; detaching everything breaks it. Training uses entropy-gated self-conditioning while inference uses unweighted posterior mean.
Illustrative example: Freeze one shortcut multiplication, not the same table's input construction path.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Why initialize calibration aψ to one and constrain it to [0.5,2]?
Designing Transformers for Continuous Diffusion Language Models
Show answer and example
Answer: One preserves initial shortcut scale. Bounded exponential calibration avoids extreme amplification or suppression, with 5000 warmup steps. Verify ranges and gradient paths.
Illustrative example: Large calibration outputs must not make the shortcut dominate the vocabulary head.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Method comparison
Should training entropy gating also weight inference posterior means?
Designing Transformers for Continuous Diffusion Language Models
Show answer and example
Answer: No. Training bootstrap confidence-attenuates and detaches feedback. Inference uses unweighted pC and no extra bootstrap. Training heuristics are not posterior guarantees.
Illustrative example: Do not discount inference probability mass a second time because training suppressed low-confidence feedback.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Basics
Why add a likelihood shortcut besides noisy embeddings?
Designing Transformers for Continuous Diffusion Language Models
Show answer and example
Answer: Codebook-to-noisy-state relations supplement token posterior with noise-conditioned backbone and bounded calibration. This is more than a normal head and needs correct gradients and self-conditioning.
Illustrative example: At low noise, nearby codebook vectors help identify the underlying token.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Core mechanism
Why mix LAMP's layer predictions in class-score space rather than concatenate features?
LAMP: Layerwise Adaptive Mixing of Predictors for Training-Free Few-Shot Vision-Language Adaptation
Show answer and example
Answer: Different layers have different widths and similarity geometry. Concatenation would introduce projection assumptions. Each LAMP branch fits support-set residuals relative to CLIP, then mixes scores in common class space. If text prototypes change, the residuals must change as well.
Illustrative example: A shallow texture branch and a deeper semantic branch each score cat versus dog before their scores are combined.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Implementation details
Why must LAMP-F update the cached response when text prototypes change?
LAMP: Layerwise Adaptive Mixing of Predictors for Training-Free Few-Shot Vision-Language Adaptation
Show answer and example
Answer: Updating prototypes changes both query priors and support residuals. The cached linear response represents how the kernel solution changes and allows exact updates without a complete solve. Updating just the query prior omits a term. Kernel statistics and mixture weights remain frozen.
Illustrative example: Moving the prototype toward cat images raises the prior for support cats, so their residual should shrink. Keeping the old correction double counts.
Sections in the Chinese source: 核心公式, 本地复现, 代码映射与边界. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Method comparison
How does a support-bank regional kernel differ from including evaluation queries in its centering statistics?
LAMP: Layerwise Adaptive Mixing of Predictors for Training-Free Few-Shot Vision-Language Adaptation
Show answer and example
Answer: Both m(x) and b use the support set as the reference; queries are scored against the fixed reference. Adding test queries would make an individual result depend on the evaluation batch. Test labels must not select mixing weights.
Illustrative example: With fixed support, query A uses the same calibration alone or with query B; B does not redefine the support mean.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Basics
Relative to what does LAMP compute its label residuals?
LAMP: Layerwise Adaptive Mixing of Predictors for Training-Free Few-Shot Vision-Language Adaptation
Show answer and example
Answer: Subtract the original CLIP text-prototype scores from true support labels. Each layer's kernel predictor corrects the prior rather than relearning all classes. If prototypes change, the residual must follow or the correction no longer matches its target.
Illustrative example: If a cat support image has a low CLIP cat score, the kernel branch learns the difference rather than retraining the vision encoder.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Core mechanism
Why doesn't CUDA-to-HIP compatibility establish performance portability?
Show answer and example
Answer: Running code does not align compute units, memory hierarchy, tiles, and parallelism. The work co-designs model, memory, parallel strategy, and kernels and estimates tail stalls. Hardware constants and optima vary.
Illustrative example: A tile filling one GPU's compute units can be limited by shared memory or tail imbalance on another.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Implementation details
Can one A100 or A30 reproduce the MI350X/B200 production system?
Show answer and example
Answer: No. The original uses matching clusters and private workloads. This is a literature and cost review, not a reproduction. A public operator would need its own hardware and acceptance protocol.
Illustrative example: A small A100 GEMM cannot verify another cluster's end-to-end training throughput.
Sections in the Chinese source: 本地复现, 复现边界. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Method comparison
Do predicted tile cycles and end-to-end throughput measure the same thing?
Show answer and example
Answer: The analytical model estimates compute, hierarchical memory, pipelines, and idle tail work. End-to-end results include actual workload and system effects. Neither constants nor local optima transfer automatically.
Illustrative example: A faster GEMM tile may be offset by synchronization or other stages.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Basics
Why is Meta/AMD hardware diversity a co-design report?
Show answer and example
Answer: It changes model, parallelism, memory strategy, GEMM and kernel porting together for recommender training. Performance depends on the combined system, not HIP conversion alone.
Illustrative example: A compiling operator can still use the wrong tile or memory layout.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Core mechanism
Why can a video generator support 4D perception without multistep denoising?
Gen4P: Video Generators are 4D Perceivers
Show answer and example
Answer: Gen4P reads motion and geometry representations inside a pretrained DiT. Clean video latents get one prescribed conditioned encoding; queries read tracks or depth. Frozen geometry adapters and trainable task LoRA differ. Supervision and decoders remain necessary.
Illustrative example: Train a head to query a point's 3D location after three seconds without generating another video.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Implementation details
What distinctions matter for continuous time queries and DiT input?
Gen4P: Video Generators are 4D Perceivers
Show answer and example
Answer: Query time interpolates endpoint embeddings with t/(T−1). DiT reads clean z0 under the final scheduler time and an empty text condition. Query time is not diffusion noise time, and this is not iterative denoising. Local work is documentation only.
Illustrative example: A midpoint time query interpolates embeddings; it does not add half-strength noise to the frame.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Method comparison
How do frozen depth-LoRA and trainable 4D-LoRA differ from full fine-tuning?
Gen4P: Video Generators are 4D Perceivers
Show answer and example
Answer: Freeze the base DiT and depth LoRA; train the 4D LoRA, queries, and decoders. This preserves generation and depth priors more directly than updating all weights. It does not guarantee every metric improves; the paper reports a depth regression.
Illustrative example: Train a temporal tracking adapter without modifying the depth update, then test both tasks.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Basics
Why reuse video generation knowledge for 3D and 4D perception?
Gen4P: Video Generators are 4D Perceivers
Show answer and example
Answer: Generation pretraining can learn appearance, motion, and geometry priors relevant to depth, tracks, and cameras. Gen4P extracts a clean-video scene representation and queries properties. It still needs geometry targets or task validation, not new video generation.
Illustrative example: Given a moving-camera clip, query a point's 3D trajectory instead of asking for another clip.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Core mechanism
Why calibrate on the deployed 1.5 approximation to √2 in FPoT?
Fractional Power-of-Two Quantization for Efficient and Effective Multiplier-Free LLM Inference
Show answer and example
Answer: Exact half-exponent calibration followed by approximate execution adds unoptimized error. Choose scale and phase for the real shift-add grid. Integer shifts round, and activation/KV formats are separate.
Illustrative example: A code calibrated at √2×scale runs at 1.5×scale; awareness lets calibration choose a better code or scale.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Implementation details
Why subtract one from nonzero magnitude code m?
Fractional Power-of-Two Quantization for Efficient and Effective Multiplier-Free LLM Inference
Show answer and example
Answer: Zero has a reserved code and m=e+1 otherwise. Recover e=m−1 for parity and shift. Signed right shifts can round, so do not claim exact real multiplication by 1.5.
Illustrative example: Magnitude code one is e=0, not an odd exponent.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Method comparison
Do FPoT weights, activations, and KV use one format?
Fractional Power-of-Two Quantization for Efficient and Effective Multiplier-Free LLM Inference
Show answer and example
Answer: No. Weights use a half-octave grid; activations and KV use uniform integers. Account for each format and execution cost separately.
Illustrative example: Shift-add weight codes do not remove activation scaling rules.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Basics
Why insert half octaves in a power-of-two grid?
Fractional Power-of-Two Quantization for Efficient and Effective Multiplier-Free LLM Inference
Show answer and example
Answer: Half-exponent spacing increases four-bit resolution, while odd exponents use 1.5 for shift-add. This trades accuracy and execution rather than allowing arbitrary floating values.
Illustrative example: A level near 1.5 is finer than only one or two.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Core mechanism
Why must role teachers score the same student tokens in segmented distillation?
EgoP²C: Internalizing Plan-Perceive-Compose for Spatial Reasoning in Egocentric Video
Show answer and example
Answer: Actual student interaction defines visited prefixes. Role-specific teachers score those prefixes to identify planning, perception, or direct-answer errors. A separate teacher-written answer loses on-policy supervision. Shared video KV must exclude role-private history.
Illustrative example: If a student asks for a viewpoint then makes a spatial judgment, teachers score the corresponding segments of that same trace.
Sections in the Chinese source: 背景与主要改动, 核心公式与信息边界. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Implementation details
Why normalize role length over the whole trajectory?
EgoP²C: Internalizing Plan-Perceive-Compose for Spatial Reasoning in Egocentric Video
Show answer and example
Answer: A role may have multiple segments; divide by all supervised tokens for that role in the trajectory, not each segment separately. Otherwise splitting a segment increases role weight. Detach teacher and old-student log probabilities, keep private decoder KV separate, and do not interpret random-teacher diagnostics as spatial capability.
Illustrative example: Ten PC tokens in two five-token segments should keep the same total weight. Separate means would double their contribution.
Sections in the Chinese source: 核心公式与信息边界, 本地复现. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Method comparison
Do the PC, V, and general G experts share the same parameter training boundaries?
EgoP²C: Internalizing Plan-Perceive-Compose for Spatial Reasoning in Egocentric Video
Show answer and example
Answer: Experts initialize from copies of the base FFN. G's FFN stays frozen, while its attention residual trains only in MOPD. PC and V supply trainable planning and perception paths. This preserves general answering without updating three full models alike.
Illustrative example: Keeping G's FFN does not forbid all adaptation; its attention residual can train later.
Sections in the Chinese source: 背景与主要改动, 核心公式与信息边界. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Basics
Why improve both question planning and perceptual answers in egocentric spatial reasoning?
EgoP²C: Internalizing Plan-Perceive-Compose for Spatial Reasoning in Egocentric Video
Show answer and example
Answer: Complex questions need decisions about where to look and what to ask, followed by reliable evidence extraction. Errors in either role propagate. EgoP²C improves both while respecting their information boundaries; final conclusions still need validation.
Illustrative example: For a drawer relative to a door, plan a viewpoint and subquestion; a left/right perceptual error cannot be fixed by fluent composition.
Sections in the Chinese source: 背景与主要改动, 核心公式与信息边界. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Core mechanism
Why consider value versus current output rather than only attention weight in DropKV?
DropKV: Decoupling Residual-Output Perturbation for Near-Optimal KV-Cache Eviction
Show answer and example
Answer: Removing a term removes its value and renormalizes the rest. Change depends on p/(1−p) times output–value difference. Unusual low-weight values can matter more than redundant high-weight ones; combining single scores is approximate.
Illustrative example: Two tokens both weighted 0.1 can cause very different changes if one value equals the aggregate.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Implementation details
Does an epsilon preserve the formula when p_j rounds to one?
DropKV: Decoupling Residual-Output Perturbation for Near-Optimal KV-Cache Eviction
Show answer and example
Answer: No. p_j/(1−p_j) is singular, and local code rejects it. Check infinite recent protection, MaxPool, and averaging GQA scores into physical KV heads.
Illustrative example: Epsilon makes a finite approximation, not the exact zero-denominator result.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Method comparison
Does exact single removal guarantee optimal multiple removal?
DropKV: Decoupling Residual-Output Perturbation for Near-Optimal KV-Cache Eviction
Show answer and example
Answer: No. Joint removal renormalizes and interacts. Per-item ranking approximates the combination, and theoretical guarantees have conditions.
Illustrative example: Two compensating values can each seem unimportant yet jointly change the output.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Basics
Why score removal by output perturbation?
DropKV: Decoupling Residual-Output Perturbation for Near-Optimal KV-Cache Eviction
Show answer and example
Answer: Weight alone omits value geometry. Estimate each removal's output change and aggregate across observed queries for eviction.
Illustrative example: A high-weight value close to the output may be safe to remove.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Core mechanism
How can BitSparse save activations losslessly without recomputing forward?
BitSparse: Compact Activation Storage for ReLU Feed-Forward Networks
Show answer and example
Answer: ReLU-family activations contain exact zeros. Store only nonzero values, a bit per position, and tile prefixes. Reconstruct for backward without quantizing values or rerunning forward. Savings depend on zeros and metadata.
Illustrative example: For eight entries with six zeros, store eight bits and two original-precision values.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Implementation details
Why not use one byte of bool per mask element?
BitSparse: Compact Activation Storage for ReLU Feed-Forward Networks
Show answer and example
Answer: The format uses one bit per position plus tile prefix and positives. Byte masks change storage accounting. Check padding, zero sign, and backward formulas such as the 2H factor in ReLU².
Illustrative example: A 32-position mask should occupy around four bytes, not 32.
Sections in the Chinese source: 核心公式, 数据协议与运行. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Method comparison
How does lossless packing differ from checkpointing or quantization?
BitSparse: Compact Activation Storage for ReLU Feed-Forward Networks
Show answer and example
Answer: Checkpointing trades recomputation for memory; quantization trades precision. BitSparse exploits exact zeros. Sparse metadata and total training memory still matter.
Illustrative example: With few zeros, metadata can offset packing gains.
Sections in the Chinese source: 背景与主要改动, 结果与独立对照. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Basics
Why does BitSparse suit exact ReLU zeros?
BitSparse: Compact Activation Storage for ReLU Feed-Forward Networks
Show answer and example
Answer: A bitmask stores positions, positive values are packed, and tile prefixes locate data. Reconstruction is lossless without approximating nonzero values.
Illustrative example: Two positives among ten positions need two values and locations, not ten floats.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Core mechanism
Why can repeated vision loops change representations when encoder weights are frozen?
AdaLoop: Thinking with the Vision Encoder
Show answer and example
Answer: Frozen parameters do not freeze hidden state. Each loop feeds updated state into a shared block modulated by the question vector. Deep supervision trains different depths, and deployment stopping must compare predictions without reading gold.
Illustrative example: A question about objects and another about occlusion can direct the same encoder toward different details.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Implementation details
Can stable-answer stopping inspect the gold label?
AdaLoop: Thinking with the Vision Encoder
Show answer and example
Answer: No. Deployment checks whether successive predictions agree. Training supervises several depths with positive weights summing to one. The local C interface also requires externally supplied detached draft embeddings and a fixed separator; it does not generate drafts itself.
Illustrative example: Two consecutive predictions of “three” can trigger stopping even when the true answer is “four.”
Sections in the Chinese source: 核心公式, 本地复现, 代码映射与边界. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Method comparison
How do vision loops differ from adding language-model reasoning tokens?
AdaLoop: Thinking with the Vision Encoder
Show answer and example
Answer: Extra language steps operate in the text stream. AdaLoop spends compute on question-conditioned visual encoding with shared weights and updated visual state. Compare matched training and budgets before claiming superiority.
Illustrative example: Counting occluded objects can revisit vision features instead of only extending text reasoning after a single encoding.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Basics
Should visual AdaLoop use identical modulation for different questions about the same image?
AdaLoop: Thinking with the Vision Encoder
Show answer and example
Answer: No. The pooled question vector conditions the visual block on the evidence each question needs. Loops add processing opportunities, not a guarantee of locating details. Shared images do not imply identical visual processing.
Illustrative example: Counting cups and reading text on them can produce different modulation vectors.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Core mechanism
Why does a tool trie use the complete within-turn call prefix instead of just the previous tool?
TRACER: Trajectory-Aware Tool Retrieval for LLM Agents
Show answer and example
Answer: The same previous tool can occur in distinct workflows. A full prefix preserves path identity and retrieves real training transitions. Context similarity adds current intent, with learned weight and temperature. Unknown prefixes fall back to catalog search rather than force an unseen path onto the nearest old branch.
Illustrative example: Example: Search→download may lead to parsing, whereas authenticate→download may need permission checking. Download alone conflates them.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Implementation details
Why must cross-fitting exclude the target trajectory's entire fold rather than only hide the current label?
TRACER: Trajectory-Aware Tool Retrieval for LLM Agents
Show answer and example
Answer: Other steps from that trajectory can affect trie counts and edge-context means. Hiding the label alone still leaks its structure. Each signal must use an index excluding the full fold, and standardization is fitted on training folds. Training real heads on synthetic vectors is not real tool-selection capability.
Illustrative example: Example: The target tool may be absent from loss input yet already counted in the all-data index, revealing its own answer path.
Sections in the Chinese source: 核心公式, 本地复现, 代码映射与边界. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Method comparison
Why learn the mix of frequency and context priors rather than simply add them?
TRACER: Trajectory-Aware Tool Retrieval for LLM Agents
Show answer and example
Answer: Frequent tools can be wrong for current context, while semantic transitions may have few samples. Two linear heads use five state features to set mixing weight and temperature. Temperature changes the confidence gate, not order. Coverage and calibration still matter; frequency is not success probability.
Illustrative example: Example: A prefix usually leads to A, but the current context resembles past transitions to B. Reduce the frequency weight rather than copy blindly.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Basics
Why can a large tool catalog consume context without surfacing the right next action?
TRACER: Trajectory-Aware Tool Retrieval for LLM Agents
Show answer and example
Answer: Keeping every description is costly, while static Top-K may omit the next interface. TRACER predicts a subset from executed transitions and current context so discovery adapts to progress. Candidate prediction does not replace argument validation or successful calls.
Illustrative example: Example: After identity lookup, account retrieval is more likely than rereading a thousand tools. If the user cancels, context should change the recommendation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Core mechanism
Why isn't recording the first unmet obligation a completed causal attribution?
Show answer and example
Answer: Obligation ordering provides a consistent label, but later failures can coexist. Hold task facts fixed and manipulate catalog, menu, workflow, and return state to study pressure. Truth-assisted routed menus are diagnostic, not oracle-free deployment ability.
Illustrative example: Failure to retrieve a tool is labeled first, but argument logic may also be wrong. Fixing retrieval may expose it; label order is not causal proof.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Implementation details
Can missing retrievals and infrastructure failures be dropped from the catalog-pressure ITT denominator?
Show answer and example
Answer: No. Keep them in the planned denominator; conditioning on available requests inflates selection rates. Distinguish first unmet obligation from separately logged termination reasons. Neither automatically establishes causation.
Illustrative example: If only 80 of 100 planned requests obtain a menu, still report outcomes over all 100.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Method comparison
How does a truth-assisted routed menu differ from an oracle-free production router?
Show answer and example
Answer: Truth-assisted narrowing is a controlled condition. It localizes bottlenecks but does not show that a live system can autonomously find the same menu.
Illustrative example: Showing only the already-known correct tool is different from retrieving it from a large catalog at deployment.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Basics
Why does ToolBottleneckBench control tool catalogs, menus, workflows, and return encoding?
Show answer and example
Answer: Tool-agent failure may reflect interface and budget interactions rather than only model capability. Hold task facts and decisions fixed and manipulate R/V/W/D separately to identify pressure associated with outcomes.
Illustrative example: A longer visible menu can raise selection costs for the same model without proving weaker reasoning.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Core mechanism
Why does TB-GAE decay by interaction turn rather than observation text length?
Terminal-RSI: Recursive Self-Improving Terminal Agents via Recursive Self-Rewrite and Policy-Refine
Show answer and example
Answer: Longer tool output does not imply more elapsed decision steps. Treating a long observation as many recurrence nodes would arbitrarily over-discount credit for earlier actions. Recurrence runs only over generated tokens, with no decay within a turn and discounting across turns. Chunk continuation requires both value and advantage; a computational boundary is not an environment termination.
Illustrative example: Two commands can produce the same outcome with ten lines or a thousand lines of logging. Verbosity alone should not reduce earlier actions' reward credit.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Implementation details
Why isn't a value bootstrap alone enough when GAE is computed in chunks?
Terminal-RSI: Recursive Self-Improving Terminal Agents via Recursive Self-Rewrite and Policy-Refine
Show answer and example
Answer: A chunk boundary does not terminate the environment. Pass both the next value and next advantage to retain the tail of the trace; emit terminal reward only once. Observations are excluded from the recurrence. The local implementation is a numeric operator without actor, critic, RSR, or real terminal training. Equivalence in the unit-clock case does not show that TB-GAE was deployed online.
Illustrative example: If the tail advantage is reset after splitting at the eleventh generated token, credit from later rewards no longer propagates backward equivalently, even with a correct next value.
Sections in the Chinese source: 核心公式, 本地复现, 代码映射、成本与边界. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Method comparison
Why do SPR and the plain partial-score FDR method rank outcomes differently?
Terminal-RSI: Recursive Self-Improving Terminal Agents via Recursive Self-Rewrite and Policy-Refine
Show answer and example
Answer: FDR scales partial scores by a family divisor and has no fixed upper bound, so a high-partial-score failure can outrank a success. SPR adds strict success and constrains the partial component below one, keeping failures below strict successes. This ranking property is not a proof about policy-gradient variance or potential invariance, and the paper's overall training gains cannot be attributed to this formula alone.
Illustrative example: A high-scoring incomplete attempt can exceed one with FDR. With SPR, it stays below the minimum score of any strict success.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Basics
Why do long terminal-agent tasks present a harder reinforcement-learning feedback problem than single-turn answers?
Terminal-RSI: Recursive Self-Improving Terminal Agents via Recursive Self-Rewrite and Policy-Refine
Show answer and example
Answer: Terminal tasks include commands, long observations, and state changes across turns. Success is often known only at the end, making it hard to assign credit to early actions; failure may also come just short of completion. Terminal RSI structures rewards and advantages around this interaction to support continued training. Partial progress is not completion, and observation length is not a direct measure of interaction time.
Illustrative example: An agent fixes the code but never runs the final test. A terminal failure label alone hides useful earlier actions; training should use evidence of progress without relaxing the completion standard.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Core mechanism
Why does SSR return an empty pool on no lexical match rather than claim a semantically similar agent?
Show answer and example
Answer: Structural signals are terms and adjacent phrases from explicit fields. IDF gives no semantic score to unmatched content. Inverted lookup skips only zero-score candidates and retains the same lexical ranking. An empty pool exposes missing evidence; arbitrary fallback is not semantic understanding by SSR.
Illustrative example: Example: A card says invoice, and a nonoverlapping paraphrase gets no hit. This is a lexical coverage gap, not a discovered synonym.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Implementation details
What constraints govern repeated words, the 64-signal limit, and empty_pool?
Show answer and example
Answer: Signals are binary and deduplicated. Frequency only affects unigram truncation above the limit, followed by deterministic bigram filling. No hit returns empty_pool without calling a provider for an invented answer. API failures and illegal indexes stay failures. There is no real local LLM orchestration success metric.
Illustrative example: Example: Repeating search ten times does not yield tenfold score, and a zero lexical match cannot silently switch to full-directory semantic retrieval.
Sections in the Chinese source: 核心公式, 本地复现, 代码映射与边界. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Method comparison
Why do inverted SSR and an exhaustive scan of the same IDF score rank positive matches identically?
Show answer and example
Answer: All IDFs are positive, so the postings union collects exactly cards sharing at least one query signal and excludes zero scores. Rank by the same score, match count, and ID tie-breaker. Savings come from skipping disjoint cards, not changing semantics. No hit means an empty pool.
Illustrative example: Example: Of a thousand cards, only ten share a term. Scoring those ten returns the exhaustive scan's positive ranking without fake fallbacks.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Basics
Why does SSR start with discovering which agent to consider in a large directory?
Show answer and example
Answer: Sending every capability description to a model raises selection cost. SSR uses discrete public-description signals to build an inverted candidate pool for later decisions. Matching is interpretable and literal, so paraphrases with no shared signal can legitimately yield nothing.
Illustrative example: Example: An invoice signal in both a task and a capability card creates a candidate. Disjoint terminology may not.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Core mechanism
Why doesn't tool-name predictability prove a guard uses the shortcut?
Shortcut Sensitivity in Agent-Safety Evaluations
Show answer and example
Answer: Label predictability shows a construction cue. Guard association is a second check. Changing only the cue under matched conditions gives stronger behavioral evidence. Control random inconsistency and placebos; association, unsigned flips, and signed effects are distinct.
Illustrative example: A name lookup can score well when dangerous samples share a tool name. To test guard reliance, rename it without changing behavior and compare decisions.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Implementation details
Why not compute F1 per interaction and then average when weighting interactions equally?
Shortcut Sensitivity in Agent-Safety Evaluations
Show answer and example
Answer: Assign steps weight 1/n_g, sum weighted TP/TN/FP/FN, then calculate F1. F1 is nonlinear, so averaging group F1s defines another metric. Signed paired effects also cannot be replaced by unsigned flip rates.
Illustrative example: Long interactions should not carry more total confusion-matrix weight just because they have more steps.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Method comparison
Do label predictability, guard-decision association, and paired intervention effects imply each other?
Shortcut Sensitivity in Agent-Safety Evaluations
Show answer and example
Answer: No. The first two motivate testing. Interventions must change only the cue while preserving risk and controlling random guard behavior and format placebos. Correlation is insufficient to establish shortcut use.
Illustrative example: A tool name can correlate with unsafe while removing it leaves guard decisions unchanged. Evidence of reliance remains insufficient.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Basics
Why does Shortcut Sensitivity first inspect construction cues in evaluation inputs?
Shortcut Sensitivity in Agent-Safety Evaluations
Show answer and example
Answer: Tool names, lengths, source markers, and repeated interactions can predict labels without genuine risk judgment. Audit label predictability, guard association, and construct-preserving paired interventions separately.
Illustrative example: If a tool name occurs in every unsafe case, a model may score well using that cue alone.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Core mechanism
Why reopen an independent session after swapping options and resample by case rather than answer?
Show answer and example
Answer: Swapping checks position stability, and separate sessions prevent the first answer from influencing the second. Both orderings still represent one project decision, not independent samples. The model sees only the pre-correction state; later records may build references but cannot be inputs or reveal human versus original LLM authorship.
Illustrative example: If plans A and B are presented in both orders in one session, the model may remember the earlier choice instead of judging again.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Implementation details
Why not attach SciPIvot's original questions file to the evaluated agent?
Show answer and example
Answer: The official source includes references. Trusted ingestion exports only the ID and two options via an allowlist, storing private grader references separately. Sessions are independent and cannot access repositories or future records to bypass options-only isolation.
Illustrative example: Providing the full questions.json with an instruction not to read references is not data isolation.
Sections in the Chinese source: 背景与主要改动, 已发布数据与来源. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Method comparison
How does swapping option order differ from treating two answers for the same case as independent samples?
Show answer and example
Answer: Both orderings measure and offset position bias, but they remain one case. Compute a case score, then resample cases. Counting answers independently inflates sample size and ignores correlation.
Illustrative example: AB and BA judgments for one case are not two distinct research decisions.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Basics
Does SciPIvot test research redirection or whether a project ultimately succeeds?
Show answer and example
Answer: It gives the pre-correction project state and two next steps and asks which is more appropriate. The reference comes from later human correction, but the tested model sees no future trajectory or option provenance. This is not prediction of eventual project success.
Illustrative example: The better next step today does not guarantee a successful project six months later.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Core mechanism
Why must all RSIFlow candidates start from the same frozen parent?
RSIFlow: Adaptive Self-Improvement Workflows with an Evolving Meta-Agent
Show answer and example
Answer: If the first candidate's changes leak into the second, improvements cannot be attributed to the selected Model, Harness, or Artifacts intervention. A shared parent and task batch provide a paired baseline. Only valid, strictly improving candidates are accepted; after a failure, retry from the original parent. Skills may come from training experience, but report-set scores must not leak into maintenance.
Illustrative example: Evaluate a prompt change and a model replacement separately against the same parent, rather than attributing the cumulative improvement of the second run to the model alone.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Implementation details
Does a CPU controller with trusted callbacks demonstrate that full RSIFlow is integrated with Evolve?
RSIFlow: Adaptive Self-Improvement Workflows with an Evolving Meta-Agent
Show answer and example
Answer: No. The callbacks and structural evidence verify parent protection, paired acceptance, failure retention, and separation of training from reporting. There is no built-in LLM, arbitrary program execution, or six-benchmark training. Declared edit scopes and high numerical fixture scores cannot replace real candidate execution and promotion evaluation, and the in-process interface is not a sandbox for malicious code.
Illustrative example: A scalar proposer that changes one number and improves a score shows that a controller branch is reachable. It does not show that real model parameters or a harness were updated as in the paper.
Sections in the Chinese source: 核心公式, 本地复现, 代码映射、成本与边界. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Method comparison
What layer of future execution does each Model, Harness, or Artifacts intervention change?
RSIFlow: Adaptive Self-Improvement Workflows with an Evolving Meta-Agent
Show answer and example
Answer: Model interventions target the model; Harness interventions change how execution is organized; Artifacts interventions act on deliverables. Their scopes and inherited state differ. Comparisons must state what changed: a one-off file fix is not general policy learning, and out-of-scope changes cannot be mixed into the designated branch. The local variant uses trusted callbacks and skill memory.
Illustrative example: Fixing one report changes only that artifact. Changing a check that runs on every task may affect future tasks, but transfer still needs verification.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Basics
Why does RSIFlow first diagnose what kind of object is responsible when automated research fails?
RSIFlow: Adaptive Self-Improvement Workflows with an Evolving Meta-Agent
Show answer and example
Answer: Poor results can stem from the model, the execution process, or a defect in the current artifact. Always editing prompts or always retraining would misallocate resources. RSIFlow uses execution evidence to select an intervention related to the failure. Both the diagnosis and the candidate need actual verification: choosing a branch does not show that its bottleneck was resolved.
Illustrative example: Fix the artifact if its output file has the wrong format; change the harness if the tool flow repeatedly omits verification. Neither necessarily requires retraining model weights.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why does a proactive assistant need a separate intervention gate after retrieving relevant memories?
MemoProAgent: Toward Memory-Driven Proactive Assistance
Show answer and example
Answer: Relevance means topical overlap, not that interrupting now is warranted. The gate decides whether evidence merits intervention; the generator uses the current scene and prior speech to speak or stay silent. Timing and count matter, so correct but late or repeated reminders are not perfect success.
Illustrative example: Example: A nut allergy is relevant when the food is visible and the assistant has not warned yet. Repeating the warning creates a false trigger.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
Can PPR retrieval and the semantic gate be replaced by one open-loop flag?
MemoProAgent: Toward Memory-Driven Proactive Assistance
Show answer and example
Answer: No. PPR performs actual personalized random walks with restart on entity relations. The gate decides whether the retrieved evidence warrants intervention. Training optimizes only generator tokens, not tool-injected evidence. A local fixture is not the official evaluation on authorized long videos and cannot use a flag to supply the right utterance.
Illustrative example: Example: Retrieving a relation between a cup and drinking habits still leaves the gate to determine whether the user just drank. The relation alone is not permission to speak.
Sections in the Chinese source: 核心公式, 本地复现与边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
Why do Perfect and a Hit for saying the right thing at least once measure different behavior?
MemoProAgent: Toward Memory-Driven Proactive Assistance
Show answer and example
Answer: Perfect requires correct content within the trigger window and exactly one utterance. Hit does not rule out early, late, or repeated interruption. Any speech on a silent negative segment is a false trigger. Raising recall alone can hurt user experience; content hits do not replace timing and count evaluation.
Illustrative example: Example: An early reminder followed by another inside the window can count as a hit without being a perfect intervention.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Why must MemoProAgent make one more judgment than ordinary QA when no user asks a question?
MemoProAgent: Toward Memory-Driven Proactive Assistance
Show answer and example
Answer: QA starts with a request. A proactive assistant must also decide whether to interrupt now, what to mention, and whether it has already said it. The paper combines long-term experience with current visual context to help at the right moment rather than maximize message count. Correct reminders can still fail through bad timing or repetition.
Illustrative example: Example: Reminding someone to bring keys before leaving can help. Repeating it three times after they already picked them up does not.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Core mechanism
Why decay rewards for repeated violations of the same policy but retain raw yield?
PROBE: Learning to Audit Policy Compliance in Tool-Using LLM Agents
Show answer and example
Answer: Decay prevents the auditor from farming one flaw and encourages policy coverage. Raw yield describes actual events and can exceed 100 per 100 episodes. Report deduplicated coverage separately from event volume. A local reward implementation is not the entire auditing RL pipeline.
Illustrative example: If one conversation triggers a violation three times, give decreasing or zero subsequent reward but preserve all three events.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Implementation details
Where must PROBE reset repeated-hit counts?
PROBE: Learning to Audit Policy Compliance in Tool-Using LLM Agents
Show answer and example
Answer: n_i counts previous hits in the current episode. The first gets weight 1, the second 1/2, and later hits zero; restart each episode. Do not decay raw-yield events, and retain zero-hit and unsupported policies in denominators.
Illustrative example: Two hits in a previous episode must not turn the first hit of the next episode into zero reward.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Method comparison
Why aren't raw violation yield, deduplicated episode-policy yield, and coverage interchangeable?
PROBE: Learning to Audit Policy Compliance in Tool-Using LLM Agents
Show answer and example
Answer: Raw yield includes repeated turn-level hits and can exceed 100 per 100 episodes. Deduplicated yield removes same-policy repetitions per episode; coverage uses all declared policies as its denominator. They describe hit volume and breadth, not one success probability.
Illustrative example: Three hits of one policy in one episode give raw yield 3, deduplicated yield 1, and add only one policy to coverage.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Basics
Why does PROBE audit policy compliance separately from task completion?
PROBE: Learning to Audit Policy Compliance in Tool-Using LLM Agents
Show answer and example
Answer: Final state or harmful-text checks can miss violations during multi-turn tool use. An auditor interacts with a frozen target while a private verifier checks the full trace for business-policy compliance, customer realism, task progress, and coverage rather than tone alone.
Illustrative example: A task can look complete despite missing required confirmation or making an unrequested state change.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why doesn't an unchanged memory-provider signature prove a patch is compatible with existing memories?
MemScope: Evolving Agent Memory Architectures through Localized Code Repair
Show answer and example
Answer: A signature constrains call shape, not internal field meanings or read/write semantics. MemScope combines raw traces, runtime memory, and code, patches in a bounded transaction, and probes real formats. Failure must preserve old code and memory; insufficient evidence can also mean no patch.
Illustrative example: Example: The parameter is still named memory, but a patch changes timestamps from seconds to days. Old records can be misread even though the code passes syntax checks.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
Why must replaceOnce match exactly once, and why allow no-patch when evidence is weak?
MemScope: Evolving Agent Memory Architectures through Localized Code Repair
Show answer and example
Answer: A local patch must identify the intended logic. Replacing arbitrary duplicate text can change the wrong behavior. replaceOnce requires a unique target and respects interface identity, syntax, and edit budgets. If localization or probe evidence is weak, do not submit a patch. Engineering validity alone is not semantic improvement.
Illustrative example: Example: Two sites have identical field reads. A patch specifying only that string cannot disambiguate them, so decline instead of changing the first match.
Sections in the Chinese source: 核心公式, 本地复现, 成本与边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
What is the trade-off between an in-place patch and regenerating the whole memory provider?
MemScope: Evolving Agent Memory Architectures through Localized Code Repair
Show answer and example
Answer: A local edit preserves provider identity and reduces compatibility and format drift. Keeping the interface still does not guarantee unchanged semantics, so run real probes. A full rewrite gives more freedom but can make old memory uninterpretable. Lines of code do not measure either strategy's effectiveness.
Illustrative example: Example: Adjusting a retrieval threshold can preserve stored records. Replacing the schema without migration may silently ignore them.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Why does MemScope inspect raw trajectories, runtime memory, and provider code together?
MemScope: Evolving Agent Memory Architectures through Localized Code Repair
Show answer and example
Answer: The same failure can come from writing, retrieval, or use, which a total score cannot distinguish. Traces show actions, memory shows stored contents, and code explains the rules. Together they support a bounded repair. If evidence is insufficient, return no-patch rather than forcing edits every round.
Illustrative example: Example: A fact written but never retrieved requires a different fix than a fact never written. A wrong answer alone does not distinguish them.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Core mechanism
Why can't a builder organize its hierarchical filesystem as an answer index using known evaluation questions?
Librarian Agents: Training LLMs to Curate Long-Context Workspaces that Anticipate Future Queries
Show answer and example
Answer: The aim is a reusable workspace a frozen answerer navigates for unseen queries. Seeing questions early creates answer-oriented shortcuts instead of cross-query organization. Preserve originals for fallback; Delete must not remove raw documents. A general cost formula does not establish nonzero shaping in the main training run.
Illustrative example: Organizing by company and date supports diverse questions. Placing one anticipated answer on the front page stops the evaluation testing general organization.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Implementation details
How does Librarian preserve provenance after deleting a synthetic node?
Librarian Agents: Training LLMs to Curate Long-Context Workspaces that Anticipate Future Queries
Show answer and example
Answer: Delete archives only the synthetic node, promotes its children, and retains raw documents and historical sources. Cold load checks hashes, hierarchy, and disk content. Local work implements filesystem contracts and frozen result recomputation without training a learned builder.
Illustrative example: After removing a summary folder, original files and sources for merged subdirectories must remain discoverable.
Sections in the Chinese source: 文件系统合同与代码映射. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Method comparison
Is Librarian's general cost utility the same as the paper's main training reward?
Librarian Agents: Training LLMs to Curate Long-Context Workspaces that Anticipate Future Queries
Show answer and example
Answer: No. The general form has build, token, and read penalties, but main training sets them to zero with gamma=1, probes answerability change every eight builder turns, and checks terminal correctness. A definable cost term is not necessarily used for shaping.
Illustrative example: A token penalty in the formula does not mean this training run deducted reward for reading another file.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Basics
Why does Librarian organize a workspace before knowing future questions?
Librarian Agents: Training LLMs to Curate Long-Context Workspaces that Anticipate Future Queries
Show answer and example
Answer: A corpus may receive many queries. Hierarchical files and source references amortize organization across documents. The builder sees only the corpus; a frozen answerer later navigates it, without prewriting test answers.
Illustrative example: Organize project files before answering several users, rather than tailoring a directory to a known question.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Core mechanism
Why keep using unexecuted Shadow actions after the stronger model takes over?
LEASH: Uncertainty-Aware Weak-to-Strong Escalation and Oversight for Computer-Use Agents
Show answer and example
Answer: Handoff mitigates weak-model failure but does not prevent the strong model drifting. Weak shadow representations offer a cheap reference for periodic review without executing shadow actions. Distance to the failure reference is not calibrated failure probability or a fixed test takeover quota.
Illustrative example: Example: The strong model is about to delete while the weak shadow suggests checking permissions. Trigger review without running both actions.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Implementation details
Why calibrate the runtime gate on each task's maximum score, and when does Shadow trigger?
LEASH: Uncertainty-Aware Weak-to-Strong Escalation and Oversight for Computer-Use Agents
Show answer and example
Answer: Calibrate reference risk on maximum action score per task, keeping all observations from a task in the same out-of-fold split. Step quantiles change the overall escalation rate. After handoff, probe every three steps from the first strong action: 1, 4, 7, with at most six interventions and a stop after two changes to strong actions.
Illustrative example: Example: Starting checks only in the third interval misses first-step risk. Splitting steps from one task across train and validation leaks task features.
Sections in the Chinese source: 核心公式, 本地复现, 代码映射、成本与边界. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Method comparison
Why isn't a reference-distance score a calibrated failure probability?
LEASH: Uncertainty-Aware Weak-to-Strong Escalation and Oversight for Computer-Use Agents
Show answer and example
Answer: It compares the weak representation's Mahalanobis distances to Pass and Fail means. Higher values only indicate relative proximity to failure. Shift, covariance, and representation geometry affect thresholds, and target routing proportions are not test quotas. Calibrate on references rather than map scores directly to percentages.
Illustrative example: Example: Score two does not imply 20% failure; a different scale or task distribution can change risk at the same score.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Basics
Why does LEASH run a weaker model and hand off only in riskier states?
LEASH: Uncertainty-Aware Weak-to-Strong Escalation and Oversight for Computer-Use Agents
Show answer and example
Answer: Strong calls throughout cost more; weak-only execution may fail in critical states. Risk-triggered takeover trades task outcomes against resource use. Handoffs are not free and risk scores are not inherently probabilities. Account for complete execution and results.
Illustrative example: Example: Use a small model for routine file search and a stronger one for rare recovery states. Value depends on takeover count and actual delivery.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Core mechanism
Why calibrate delegation thresholds by decision type on the agent's own shadow trajectories?
JevAgentHarness: Benchmarking, Deploying and Training Decision Models That Share Control with LLMs
Show answer and example
Answer: Closed-loop execution reaches states after errors, which clean offline multiple choice does not cover. Tool and argument choices also have different risks and benefits. Compare decision and strong models on matched examples, choose reliable thresholds by task value and cost, and fall back on uncertain or unseen classes.
Illustrative example: Example: A small model chooses tools correctly but often supplies wrong arguments. Give the categories separate gates instead of applying one overall accuracy.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Implementation details
Should unknown classes or undersampled categories use the nearest threshold?
JevAgentHarness: Benchmarking, Deploying and Training Decision Models That Share Control with LLMs
Show answer and example
Answer: No. Default bins require at least twenty rows, keep equal confidence together, and merge undersized tails. Too few rows for a category, no eligible suffix, or an unknown type means fall back to the strong model. Fit uses calibration validation only; runtime sees no gold or strong-model answers. Local handwritten callbacks test the contract.
Illustrative example: Example: Three argument decisions cannot calibrate a reliable threshold. Fall back rather than borrowing a different tool class's threshold.
Sections in the Chinese source: 核心公式, 本地复现, 成本与边界. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Method comparison
How does a cost- and task-value threshold differ from a fixed 0.7 confidence gate?
JevAgentHarness: Benchmarking, Deploying and Training Decision Models That Share Control with LLMs
Show answer and example
Answer: Utility uses paired correctness differences between the decision and strong models plus saved call cost. Select a confidence suffix whose bins are all nonnegative. A constant ignores class, error cost, and overconfidence. Fitting is not infallible; the paper retains failed calibration and no-gain additional training.
Illustrative example: Example: A high-stakes error can outweigh a small saving, while a recoverable decision may tolerate a lower delegation threshold.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Basics
Why not use equally expensive reasoning for every small decision in Jev Agent Harness?
JevAgentHarness: Benchmarking, Deploying and Training Decision Models That Share Control with LLMs
Show answer and example
Answer: Tool workflows have bounded decisions, some suitable for a cheap decider and others needing more reasoning. Explicit delegation with fallback lets savings and error costs define the scope. A cheap decider is not automatically good for open tasks; validate it on states it actually encounters.
Illustrative example: Example: A cheap model may decide whether to keep a candidate record. An open explanation with missing evidence may still require the main model.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why freeze the reader and judge before optimizing the memory writer?
Hy-Memory-30B-A3B: Attributed Reinforcement Learning for Long-Term Memory Writing
Show answer and example
Answer: Changing reader and judge together confounds better writing with stronger reading. Holding them fixed lets chunk probes and factual attribution isolate writer contributions; a storage-growth penalty prevents keeping everything. Each rollout gets one vote per chunk. More calls are not more independent evidence.
Illustrative example: Example: Give both writers' outputs to the same reader. Differences better reflect which facts were retained than whether one writer happened to get a stronger answer model.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
Can multiple calls for the same chunk count as multiple reward votes?
Hy-Memory-30B-A3B: Attributed Reinforcement Learning for Long-Term Memory Writing
Show answer and example
Answer: No. Each rollout casts one vote per chunk, and advantages are normalized within chunk groups. Counting calls would give extra influence to more active trajectories. Gold stays with the judge, not writer, reader, or retrieval query. Large-scale writer RL was not run locally.
Illustrative example: Example: Five calls checking one chunk and one call in another rollout still contribute one vote each, not five versus one.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
Why use both per-chunk probes and final outcome rewards?
Hy-Memory-30B-A3B: Attributed Reinforcement Learning for Long-Term Memory Writing
Show answer and example
Answer: Final answer feedback is sparse. Chunk probes give nearer signals about writing quality, and cross-chunk attribution estimates factual contributions. A growth penalty discourages storing everything. High local probe scores do not ensure cross-session answers, and a final outcome alone cannot identify the useful write.
Illustrative example: Example: An early chunk saves the right date, then a later chunk overwrites it incorrectly. Local probes and final attribution need to reveal the damage together.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Why are storing every conversation and writing memories for future questions different tasks?
Hy-Memory-30B-A3B: Attributed Reinforcement Learning for Long-Term Memory Writing
Show answer and example
Answer: Raw conversations are large and later readers have limited context. A writer must preserve usable facts and relationships while controlling storage growth. Hy-Memory trains that policy rather than merely enlarging the archive. The useful test is whether later answers can be grounded in the resulting memory.
Illustrative example: Example: A discussion reschedules a meeting repeatedly. A later question needs the final date and its confirmation, not a mechanical archive of every repeated message.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Core mechanism
Why is HDA residual G only an upper bound on generalization improvement, rather than direct transfer evidence?
Harness-Delta Attribution: Dissecting Gains From Harness Evolution
Show answer and example
Answer: Compute matching removes extra computation and neutralizing detected shortcuts removes some overfit, but undetected shortcuts can remain. Controls need task-aware human review and attribution starts only after confirming a significant gain. Algebraic accounting does not establish full causal identification or transfer to a new distribution.
Illustrative example: Removing a known filename shortcut may leave gains from hidden directory ordering. Do not call all remaining gains general reasoning.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Implementation details
Can negative raw T, O, or G be clipped to zero before displaying HDA percentages?
Harness-Delta Attribution: Dissecting Gains From Harness Evolution
Show answer and example
Answer: Preserve signed raw effects satisfying Delta=T+O+G. Clip display shares only according to their separate rule and do not overwrite original attribution. All four arms must align by item and cost units, with human-reviewed control rationale.
Illustrative example: If neutralization improves scores, retain negative O instead of setting it to zero for a prettier chart.
Sections in the Chinese source: 核心公式, 本地复现与边界. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Method comparison
Why can't HDA's residual G be called proven generalization gain?
Harness-Delta Attribution: Dissecting Gains From Harness Evolution
Show answer and example
Answer: It is the difference after neutralizing detected shortcuts relative to the compute-matched baseline and may retain unidentified shortcuts, so it is only an upper bound. O covers detected and neutralizable mechanisms, a lower bound on overfit.
Illustrative example: Gains after eliminating one form of answer leakage do not rule out another shortcut.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Basics
Why does HDA first test paired gain of evolved versus baseline?
Harness-Delta Attribution: Dissecting Gains From Harness Evolution
Show answer and example
Answer: If the observed gain could be noise, decomposing it into compute, shortcuts, and residual has no reliable starting point. First perform a two-sided paired bootstrap, then build compute-matched and shortcut-neutralized controls only after significance. Both require task-aware human review.
Illustrative example: Small fluctuations across runs on the same tasks do not justify quantifying generalization.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Core mechanism
Why can't the Observer and shared notes replace each executing agent's actual environment feedback?
A team of Gemini 4 Argon agents beats Minecraft
Show answer and example
Answer: Summaries help coordinate but can lag or miss local changes. Programs must use world responses to determine success. Programmatic action, durable notes, and multi-session collaboration support long tasks together. A chat claim cannot advance dependent steps, and a described case is not a local run.
Illustrative example: Example: A note says a bridge was fixed, but another agent sees it destroyed. Update from current observation instead of trusting the old note.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Implementation details
Can a 27-hour completion be stated as 100% success or a local executable capability?
A team of Gemini 4 Argon agents beats Minecraft
Show answer and example
Answer: No. One case has no independent repetition denominator. Roughly 24,000 calls are the authors' costs. Local work records literature and cost without commercial calls, game execution, GPU receipts, or an Evolve operator. Formal interaction notation is not a Minecraft implementation.
Illustrative example: Example: One game completion is one successful case, not ten successes or a local receipt based on the authors' video.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Method comparison
How compare evidence and costs for four collaborating agents versus one long-running agent?
A team of Gemini 4 Argon agents beats Minecraft
Show answer and example
Answer: Parallel roles can divide exploration and execution but add communication, duplication, and coordination. The paper gives one long success in a random Easy world, not multiseed controls establishing reliability or cost advantage. Completion does not mean four agents generally beat one.
Illustrative example: Example: Four agents can gather resources faster or all collect the same item. One success cannot isolate teamwork from extra compute.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Basics
Why study teamwork through long-horizon Minecraft completion rather than single-instruction following?
A team of Gemini 4 Argon agents beats Minecraft
Show answer and example
Answer: Exploration, resource gathering, crafting, and combat create dependencies in time and materials, with new state along the way. Multiple agents, observation, and shared information support progress. One full completion demonstrates a possibility but not a probability or causal gain from one module.
Illustrative example: Example: Mining and food gathering can be divided across roles. A note that mining is done does not produce crafting material if it was never acquired.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Core mechanism
Why generate one memory per context–goal–budget and reuse it for all follow-up questions?
GACC-Bench: Benchmarking Goal-Adaptive Context Compression Ability across Large Language Models
Show answer and example
Answer: Compression sees only the high-level goal, not individual questions. Reuse tests whether one memory retains evidence for the group. Recompressing per question gives the compressor extra query information and changes the task. Strict token truncation, empty memory for zero budget, and no padding determine actual reader inputs, beyond nominal budget.
Illustrative example: A project-decision goal can lead to date and reason questions. Both should read the same precompressed memory, not query-specific summaries extracted later.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Implementation details
What should GACC pass to the reader when compressor output exceeds budget or the budget floors to zero?
GACC-Bench: Benchmarking Goal-Adaptive Context Compression Ability across Large Language Models
Show answer and example
Answer: Use the fixed tokenizer to take the first L tokens and decode, without regeneration or padding. L=0 gives empty memory. Bind caches to source, goal, tokenizer, and compressor version so configurations do not share memories incorrectly.
Illustrative example: With three source tokens and retention 0.2, L=0; do not retain one token to improve scores.
Sections in the Chinese source: 核心公式, 代码映射、成本与边界. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Method comparison
How can GACC Micro be high while Macro is zero?
GACC-Bench: Benchmarking Goal-Adaptive Context Compression Ability across Large Language Models
Show answer and example
Answer: Micro counts correct answers over all questions. Macro scores a goal only when all its questions are correct. Missing one key question in every group can retain high per-question accuracy but zero goal completeness.
Illustrative example: Two goals with two of three questions correct each yield Micro=2/3 and Macro=0.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Basics
Why does GACC-Bench use one compressed memory for several questions under one goal?
GACC-Bench: Benchmarking Goal-Adaptive Context Compression Ability across Large Language Models
Show answer and example
Answer: Only the high-level goal is visible during compression. Reuse checks whether the whole group's evidence was retained. Recompressing after seeing a question becomes targeted extraction for a known query.
Illustrative example: A travel-goal memory should answer both transport and lodging without creating a new summary per question.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Core mechanism
Why doesn't a satisfiable result prove that a formal specification faithfully captures the task?
Repairing LLM-Generated Specifications with Deterministic Diagnosis
Show answer and example
Answer: The solver checks only the constraints that were written. It cannot tell whether a natural-language requirement was omitted or substituted. DRAGO diagnoses vocabulary contracts, reachability probes, and alignment with the task separately. Repairs must strictly reduce blockers without gaming the check by dropping goals. A probe that runs out of budget must report unknown, not proven unsatisfiable.
Illustrative example: Example: A task requires reaching two places, but a specification listing only one can still be satisfiable. Removing the harder goal makes solving easier without repairing the specification.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Implementation details
Is it enough for a repair to leave the total blocker weight unchanged or lower?
Repairing LLM-Generated Specifications with Deterministic Diagnosis
Show answer and example
Answer: No. Blocker weight must strictly fall, no blocker dimension may be added, and the goal count must not decrease. Object counts may move only toward the public specification, and a previously solvable specification must not become unsatisfiable or unknown. On rejection the old object is kept and the loop stops. A non-increase rule can accept pointless rewrites, while goal deletion games the diagnostic.
Illustrative example: Example: Reject a rewrite that merely rewords a specification while leaving both blockers. Also reject a rewrite that reduces blockers by dropping a goal.
Sections in the Chinese source: 核心公式, 本地复现与边界. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Method comparison
Why can't delete relaxation and full BFS substitute for one another as reachability checks?
Repairing LLM-Generated Specifications with Deterministic Diagnosis
Show answer and example
Answer: Delete relaxation ignores delete effects. A goal absent at its fixed point is unreachable, but goals that appear may not be jointly achievable. Full BFS retains delete effects and proves unsatisfiability only by exhausting the state space; exhausting the budget means unknown. Relaxation is cheaper but one-sided, while BFS is stricter and limited by search size.
Illustrative example: Example: Taking a key may remove a different condition. Two goals can each appear reachable under relaxation even if they cannot hold simultaneously. Full-state search exposes that conflict.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Basics
What kind of apparent success does DRAGO guard against when an LLM converts a natural-language problem to a formal specification?
Repairing LLM-Generated Specifications with Deterministic Diagnosis
Show answer and example
Answer: The model can produce a parseable, solvable specification that omits requirements, and the solver will faithfully solve the wrong problem. DRAGO uses localized diagnostics to constrain repairs rather than taking any solution as proof the original problem was solved. Formalization and solving need separate checks.
Illustrative example: Example: A schedule must assign both people. Omitting one person makes the scheduling problem easier, but that apparent success exposes a formalization error.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Core mechanism
Why does a tool timeout neither support nor refute a root-cause hypothesis?
Diviner: Agentic RCA with Human-Governed Adaptation for Large-Scale Infrastructure
Show answer and example
Answer: A timeout means no valid observation, not that the service is healthy or faulty. Evidence must match incident, target, time window, and availability. Similar incidents are clues only. Adapting investigation is distinct from publishing a workflow or executing mitigation, which requires separate authority.
Illustrative example: Example: A ranking-log query times out. Mark unknown and gather evidence differently rather than conclude ranking was healthy or use a later postmortem as contemporaneous evidence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Implementation details
What identity and time constraints must evidence satisfy before entering EvidenceLedger?
Diviner: Agentic RCA with Human-Governed Adaptation for Large-Scale Infrastructure
Show answer and example
Answer: Bind records to the current incident, target, and investigation window. Distinguish event time from evidence availability, which must not exceed as-of. Otherwise unrelated logs or later summaries can be mistaken for evidence available then. Local tests verify provenance and governance, not root cause.
Illustrative example: Example: A predawn error summarized in the morning is not known to the predawn investigator. A similar error from another incident cannot be mixed in.
Sections in the Chinese source: 核心公式, 本地复现, 成本与边界. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Method comparison
Why govern live investigation adaptation differently from publishing persistent knowledge?
Diviner: Agentic RCA with Human-Governed Adaptation for Large-Scale Infrastructure
Show answer and example
Answer: A current plan can adapt to new evidence with local scope. Shared workflows influence future incidents and require a new version, replay validation, and artifact-owner approval. The paper does not authorize autonomous production mitigation or turn observational MTTR changes into randomized causal effects.
Illustrative example: Example: Add a config lookup to this incident. Rewriting the team-wide playbook needs independent validation and an owner release.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Basics
Why log evidence availability time as well as event time during incident diagnosis?
Diviner: Agentic RCA with Human-Governed Adaptation for Large-Scale Infrastructure
Show answer and example
Answer: An investigator may not have later aggregates or conclusions when an event happens. Diviner requires matching incident, target, and window with available time no later than as-of. That separates contemporaneous material from final explanations, but structural compliance does not prove entailment or root cause.
Illustrative example: Example: An error at 10:00 appears in a report generated at 11:00. A 10:30 investigation cannot use that report despite its earlier event time.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Core mechanism
Why subtract strong-model self-disagreement from weak–strong disagreement?
Disagreement-Guided Weak-to-Strong Routing
Show answer and example
Answer: Disagreement may come from strong-model instability rather than a fixable weak answer. Estimate self-disagreement over distinct sample indices and subtract it for a better escalation signal. The score can be negative and is not a success probability. At test time run only weak representations and probes before paying for escalation.
Illustrative example: Opposite strong answers make high weak–strong disagreement inconclusive. A consistent strong model disagreeing with the weak one provides a clearer signal.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Implementation details
Why exclude self-pairs when estimating strong-model self-disagreement?
Disagreement-Guided Weak-to-Strong Routing
Show answer and example
Answer: Use K_s(K_s−1) ordered pairs of distinct indices. Diagonal comparisons always agree and bias disagreement down. Calibrate inference thresholds on validation only; under test distribution shift, exact routing quotas are not guaranteed.
Illustrative example: Two strong responses give two ordered cross pairs, not four pairs including both self-pairs.
Sections in the Chinese source: 核心公式, 代码映射与边界. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Method comparison
How does the corrected routing score range differ from raw weak–strong disagreement probability?
Disagreement-Guided Weak-to-Strong Routing
Show answer and example
Answer: Raw disagreement is a probability. The corrected score subtracts two probe outputs, can be negative, and must not be clipped to [0,1]. Take the minimum response-level prediction across weak answers first, then subtract prompt-level strong self-disagreement.
Illustrative example: Weak–strong disagreement 0.2 and strong self-disagreement 0.5 yield −0.3.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Basics
Why doesn't weak-model uncertainty necessarily justify escalation?
Disagreement-Guided Weak-to-Strong Routing
Show answer and example
Answer: If the strong model is also unstable on the question, escalation may not improve reliably. Subtract strong self-disagreement from weak–strong disagreement to learn escalation value. Offline equivalence comparisons need no correct-answer labels.
Illustrative example: Both models may waver between options. The weak model's hesitation alone does not show that the strong model can fix it.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Core mechanism
Why is retraining from the same base each round important for attributing the effects of data research?
AutoSFT: An Agentic System for Automated Data Research in Supervised Fine-Tuning
Show answer and example
Answer: This makes cross-round changes primarily attributable to the data pool, mix, and order, rather than cumulative learning from the previous round's weights. Fix the trainer and budget and register intervention hypotheses to compare whether the predicted mechanisms appear. Proposals from different analytical views are not completed experiments, and predictions must not be rewritten after seeing the scores.
Illustrative example: If round two repairs examples with incorrect answers, retrain from the same base. Continuing from round one's weights would confound the change with extra optimization steps.
Sections in the Chinese source: 背景与主要改动, 核心公式：固定训练合同的形式化. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Implementation details
Why preregister hypotheses before training, and why retain an independent external holdout after decontamination?
AutoSFT: An Agentic System for Automated Data Research in Supervised Fine-Tuning
Show answer and example
Answer: Record the change, control, expected direction, and evidence in advance to prevent rewriting predictions after the fact. Repeatedly using the same evaluation failures and choosing the best score can adapt to that evaluation even without direct n-gram duplication. Only a separate, frozen holdout offers a distinct test of generalization. The local project did not run the eight-H800 training or the paid research campaign and supplies no simulated training metrics.
Illustrative example: If the score increases but the predicted evidence does not appear, report that the hypothesis was not confirmed. Do not revise the earlier prediction to match the outcome.
Sections in the Chinese source: 核心公式：固定训练合同的形式化, 成本决策与本地状态. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Method comparison
How do four analytical views differ from a single agent adjusting only dataset proportions?
AutoSFT: An Agentic System for Automated Data Research in Supervised Fine-Tuning
Show answer and example
Answer: Data, failures, training signals, and past research provide different evidence. Execution subagents return traceable artifacts and the Meta Agent chooses interventions beyond changing the mixture. The authors' six action categories were identified retrospectively rather than used as a fixed menu; a table of rule-based options is not open-ended data research.
Illustrative example: Failure analysis may suggest correcting answers, while training signals point to ordering. Compare the two proposals using evidence, rather than increasing one category's share by five percentage points every round.
Sections in the Chinese source: 背景与主要改动, 核心公式：固定训练合同的形式化. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Basics
Why does AutoSFT make the selection and organization of training data an object of automated research?
AutoSFT: An Agentic System for Automated Data Research in Supervised Fine-Tuning
Show answer and example
Answer: Different data contents, quality levels, and combinations can produce different capabilities from the same base, while manually trying many recipes is costly. AutoSFT proposes and tests data hypotheses rather than changing a training hyperparameter once. To understand a data change, control training conditions and isolate evaluation; rising development scores do not establish final generalization.
Illustrative example: If formatting errors may stem from a shortage of examples, add those examples and compare from the same base. Reusing previous-round weights introduces extra training history.
Sections in the Chinese source: 背景与主要改动, 核心公式：固定训练合同的形式化. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Core mechanism
Why does an independent training service also need frozen evaluation and checkpoint lineage?
Athanor: An Agent-Native System for Autonomous Post-Training Research
Show answer and example
Answer: Asynchronous experiments produce many combinations of data, algorithms, and stages, which filenames alone cannot identify. Lineage makes parent-child relations traceable, frozen evaluation limits drift as search progresses, and a final held-out set isolates the researcher's selection. A single highest development score can be noise, not evidence of robust capability gains.
Illustrative example: For a model that underwent SFT, merging, and then self-distillation, missing parent checkpoints make it difficult to explain which intervention changed the score or to reproduce the candidate.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Implementation details
Why can't the highest development score found during search serve as the final formal result?
Athanor: An Agent-Native System for Autonomous Post-Training Research
Show answer and example
Answer: Selecting the maximum over repeated proposals on a noisy development set introduces selection bias. Formal comparisons must distinguish the campaign, submitted multi-seed averages, and reevaluation under consistent decoding; the final held-out set is hidden from researchers. The local entry is a literature and cost review only, with no automated research run, model execution, or GPU receipt.
Illustrative example: One chance high score in a hundred evaluations does not establish a checkpoint's expected ability. Freeze selection and reevaluate under an independent protocol.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Method comparison
If external teachers are prohibited, how does self-distillation differ from using a stronger commercial teacher?
Athanor: An Agent-Native System for Autonomous Post-Training Research
Show answer and example
Answer: Self-distillation uses permitted models and existing data within the system. An external teacher introduces additional capability, cost, and data provenance, violating the research contract. A commercial model may power the research agent, but its answers must not be slipped into training labels. Keep research control separate from supervision of the model being trained.
Illustrative example: A commercial agent may plan the next round, but adding its high-quality solutions directly to the dataset no longer satisfies the no-external-teacher setting.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Basics
What is the role of separating the research agent from the asynchronous training and evaluation service?
Athanor: An Agent-Native System for Autonomous Post-Training Research
Show answer and example
Answer: The agent selects data, algorithms, and stages. The service trains, decontaminates, runs frozen evaluation, and saves checkpoint lineage, connecting proposals to real artifacts. Separation also helps restrict access to the final held-out set. Development scores returned by the service remain noisy and are not the true performance of every proposal.
Illustrative example: After the agent proposes self-distillation, wait for the service to complete and record the new checkpoint and evaluation. A proposal saying 'expected to improve' is not an outcome.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Core mechanism
Why independently verify unchanged goals and solvability after perturbations pass JSON checks?
AgentPerturb: Evaluating LLM Agent Robustness under Controlled Interaction Perturbations
Show answer and example
Answer: Parseable inputs can still change the task or remove essential information. Single-target semantic validation holds other factors, backend, and oracle fixed so differences reflect interaction pressure. Report paired coverage and bounds for missing scores rather than dropping failures and claiming robustness.
Illustrative example: Rewording a tool description should preserve behavior. Removing the only required tool introduces unsolvability, not the same perturbation effect.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Implementation details
Can AgentPerturb drop unscored requests when calculating paired degradation?
AgentPerturb: Evaluating LLM Agent Robustness under Controlled Interaction Perturbations
Show answer and example
Answer: Compute C, P, and Delta on scored pairs, but report coverage and unknown bounds across all requested pairs. Missing is unknown, and native partial credit must not be arbitrarily binarized to align formats.
Illustrative example: If only 60 of 100 planned pairs have both scores, do not report just their drop while hiding 40 missing pairs.
Sections in the Chinese source: 核心公式, 本地复现. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Method comparison
What do AgentPerturb syntax checks and independent semantic verification establish?
AgentPerturb: Evaluating LLM Agent Robustness under Controlled Interaction Perturbations
Show answer and example
Answer: Syntax checks parseability, not goal preservation or solvability. An independent verifier checks single-target change, unchanged goal, solvability, and executability together with a native validator. A local arithmetic verifier does not prove arbitrary natural-language equivalence.
Illustrative example: Valid JSON that changes a sum into a product changes the goal and is not a robustness perturbation.
Sections in the Chinese source: 背景与主要改动, 真实性、gold 隔离与成本. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Basics
Why does AgentPerturb change only one of U, P, C, or T at a time?
AgentPerturb: Evaluating LLM Agent Robustness under Controlled Interaction Perturbations
Show answer and example
Answer: User request, procedure, history/context, and tool contract are distinct input surfaces. Holding others, backend, and success oracle fixed isolates pressure better. Changing several surfaces introduces confounds.
Illustrative example: A single-target pair can change only tool wording while preserving user requirements and tool implementation.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Core mechanism
Why does CASI freeze retrieval snapshots and recombine subsets rather than split the set reward evenly?
CASI: Credit Assignment via Set Intervention for Agentic Retrieval
Show answer and example
Answer: Actions can complement or substitute for each other. CASI retrieves once per action, computes marginal value across subsets of fixed results, and shapes advantages with Shapley credit. This avoids repeated-retrieval noise but does not establish an identical unbiased gradient to the original set objective.
Illustrative example: If rewrites A and B retrieve the same products and C adds a new category, averaging overstates redundant actions; fixed-set recombination exposes C's contribution.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Implementation details
Why jointly project CASI's bounded zero-sum advantages instead of clipping dimensions independently?
CASI: Credit Assignment via Set Intervention for Agentic Retrieval
Show answer and example
Answer: Independent clipping breaks zero sum and can shift the mean action advantage away from the original rollout advantage. Joint projection solves for a shared offset λ before clipping to satisfy both bounds and zero sum. It preserves the arithmetic action mean, not a token-length-weighted mean, and does not guarantee the original unbiased gradient.
Illustrative example: Clipping (2, −1, −1) to ±1 yields a negative sum. A common offset before clipping can meet both constraints.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Method comparison
Why are Singleton and LOO generally insufficient to recover CASI's Shapley credit with four or more actions?
CASI: Credit Assignment via Set Intervention for Agentic Retrieval
Show answer and example
Answer: Singleton observes individual values and leave-one-out observes removal from the full set. Shapley averages marginal contributions across all coalitions, including intermediate subsets with higher-order cooperation. Equal endpoint values do not fix intermediate values; additive cases need no extra intervention.
Illustrative example: Two queries may be useless alone but jointly cover a need, and a third can replace one. Singles or full-set removals can miss the dependency.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Basics
Why does CASI separate one retrieval per action from re-evaluation of subsets?
CASI: Credit Assignment via Set Intervention for Agentic Retrieval
Show answer and example
Answer: Reretrieval can change action results across subsets, mixing backend randomness with cooperation effects. Freeze action candidate snapshots first and then recombine them to compare marginal contributions over identical evidence. State assumptions about evaluation backend sensitivity.
Illustrative example: Removing one of two equivalent queries may leave the set unchanged. If reretrieval changes the results, redundancy and retrieval fluctuation are no longer distinguishable.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Core mechanism
Why generate fresh replays and require improvement rather than distill old failures?
Failure Is the Mother of Success: Recovering Supervision with Grounding Hindsight
Show answer and example
Answer: Verify the note with actual new behavior and a strict pass-rate gain. Uniformly sample accepted replays and compare note/no-note positive lift on identical prefixes. Old failures aren't fixed by explanation; no gold in notes and no auxiliary term without acceptance.
Illustrative example: A convincing postmortem with all-failed new attempts is not valid teacher guidance.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Implementation details
Do zero-lift RSHD tokens count in the auxiliary denominator?
Failure Is the Mother of Success: Recovering Supervision with Grounding Hindsight
Show answer and example
Answer: Yes. Count all valid accepted replay tokens, including zeros. No accepted note gives exactly zero auxiliary loss. Detach teacher/lift and never construct notes from gold.
Illustrative example: Dividing only by positive-lift count would overamplify sparse weights.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Method comparison
How does uniform replay selection differ from keeping successes only?
Failure Is the Mother of Success: Recovering Supervision with Grounding Hindsight
Show answer and example
Answer: Uniform selection preserves the note-induced distribution. Success-only conditions on outcomes a second time after note-level gating and changes provenance.
Illustrative example: An accepted replay group can include failures; don't silently retain only the correct answer.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Basics
Why require note-induced replays to improve RSHD pass rate?
Failure Is the Mother of Success: Recovering Supervision with Grounding Hindsight
Show answer and example
Answer: Plausible reflection may not improve behavior. Independently resample and accept only strict improvement over original group, then distill new continuations rather than old failures.
Illustrative example: A note suggesting unit checks is rejected if retries still all fail.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Core mechanism
Why keep cyclic preferences instead of removing or labeling their videos bad?
When Pairwise Preferences Cycle: A Reliability-Anchored Preference Walk for Video GRPO
Show answer and example
Answer: Cycles can come from different strengths. Preserve opponent identities, edge clarity and cycle inconsistency, propagate reliably from losers to winners and adjust win-rate credit. Keep direction, convergence and entropy protocol; ordinary PageRank differs.
Illustrative example: A beats B, B beats C and C beats A across visual quality, motion and adherence; deleting all loses comparisons.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Implementation details
May nonconvergent RAP-Walk automatically add PageRank teleportation?
When Pairwise Preferences Cycle: A Reliability-Anchored Preference Walk for Video GRPO
Show answer and example
Answer: No. Report failure per protocol. Use W_ji orientation, restandardize after mixing and detach preference graph/advantages. Teleportation changes stationary credit.
Illustrative example: Reversing loser-to-winner edges can reward opposite nodes even with a normalized matrix.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Method comparison
How does reliable random walk differ from averaging wins?
When Pairwise Preferences Cycle: A Reliability-Anchored Preference Walk for Video GRPO
Show answer and example
Answer: Propagate from losers to winners with reliability, obtain stationary credit and mix with a win-rate anchor. Minimum entropy prevents excessive concentration. Average wins lack that propagation.
Illustrative example: Beating opponents of different reliability does not yield identical propagated credit.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Basics
Why retain opponent identity and preference cycles in RAP-Walk?
When Pairwise Preferences Cycle: A Reliability-Anchored Preference Walk for Video GRPO
Show answer and example
Answer: Win rate loses who was beaten and where preferences conflict. Edge clarity and cycle reliability inform graph credit. Cycles can reflect multi-dimensional tradeoffs, not poor samples.
Illustrative example: A can win on motion, B visuals and C adherence, forming a meaningful cycle.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
Why replay model-generated residual states instead of fixed residual inputs?
On-Policy Residual Learning for Diffusion Language Models
Show answer and example
Answer: Predictions become subsequent context and propagate errors. Fixed/ideal training can miss the model's inference-state distribution. Online collection and exact replay with entropy contraction address this during training, not through final-output fixes. No local full training evidence.
Illustrative example: An early wrong token bias affects the next denoising step; perfect residuals may never teach this correction.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
Can the repository call an ORL adapter for residual traces?
On-Policy Residual Learning for Diffusion Language Models
Show answer and example
Answer: No. Literature-only: no adapter, metrics or GPU receipt. Equations are explanatory; generic diffusion or a registry name does not implement online collection and exact replay.
Illustrative example: Adding an evolve key cannot create the trajectory process.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
How does ORL differ from smoothing outputs after inference?
On-Policy Residual Learning for Diffusion Language Models
Show answer and example
Answer: Postprocessing changes final outputs only. ORL feeds previous distributions into later model states and trains on them, including exact replay and entropy contraction; simple averaging is not equivalent.
Illustrative example: Averaging two answers doesn't teach denoising from a mistaken previous state.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
Why train ORL diffusion on its own induced residual contexts?
On-Policy Residual Learning for Diffusion Language Models
Show answer and example
Answer: The previous prediction creates the next residual and carries model errors at inference. Ideal residuals induce a state mismatch; online collection and replay expose encountered states.
Illustrative example: A denoising step must see wrong prior probabilities to learn recovery from them.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Core mechanism
Why budget with absolute gain but update queries with signed gain?
Information-Gain Calibrated On-Policy Distillation for Deep Search Agents
Show answer and example
Answer: Large positive or negative changes deserve supervision; action credit must distinguish help from harm. The additive parts act on different sets. Answers belong to frozen training probes only; long final replies must not absorb short-query budgets.
Illustrative example: Misleading results can justify large correction but negative rather than positive direction.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Implementation details
Why is IQC outside ITC, and what does λ=0 mean?
Information-Gain Calibrated On-Policy Distillation for Deep Search Agents
Show answer and example
Answer: Total action credit is warm-up coefficient times gain, not multiplied again by turn budget. λ=0 gives equal turn budget, not equal token weight. Final response gain is zero but base distillation remains.
Illustrative example: A short call and long answer with equal budgets have different token weights.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Method comparison
Do ITC and IQC change credit magnitude or direction?
Information-Gain Calibrated On-Policy Distillation for Deep Search Agents
Show answer and example
Answer: ITC budgets turns by absolute gain; IQC uses signed gain to calibrate action tokens. Large negative gain merits attention, not positive reinforcement.
Illustrative example: Misleading retrieval that lowers confidence needs correction, not reward.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Basics
Why judge IG-OPD queries after retrieval?
Information-Gain Calibrated On-Policy Distillation for Deep Search Agents
Show answer and example
Answer: Teacher phrasing preferences can dislike effective searches. Changes in answer support after retrieval indicate useful information. Answer probes are training-only.
Illustrative example: An awkward query retrieving key evidence can be valuable despite its wording.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
Why can't GGPO behavior and reference be treated as the same role?
Gibbs-Gap Policy Optimization for Off-Policy RL with Verifiable Rewards
Show answer and example
Answer: Behavior is the sample source; reference defines KL-anchored Gibbs objective. Optimize adjusted log-mean-exp vs mean without extra behavior IS. Reference refresh cannot rewrite sampling provenance; filtered/normalized logs are not raw theorem quantities.
Illustrative example: A batch collected by an old policy remains old-policy data after changing the optimization anchor.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
Why not backpropagate directly through clipped GGPO LME?
Gibbs-Gap Policy Optimization for Off-Policy RL with Verifiable Rewards
Show answer and example
Answer: Compute frozen coefficients with clipped learner/reference ratios, then update original sequence logp. Backpropagating through clamp zeroes saturated gradients and changes the surrogate. Keep full-batch normalization/filtering.
Illustrative example: Clipping weight construction doesn't remove saturated positions from learning.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
How does GGPO's Gibbs gap differ from learner/behavior importance weighting?
Gibbs-Gap Policy Optimization for Off-Policy RL with Verifiable Rewards
Show answer and example
Answer: Differentiate adjusted-reward log-mean-exp minus mean within groups. No extra learner/behavior ratio multiplies updates. Off-policy data alone do not justify adding one.
Illustrative example: Old-policy data do not imply every objective must use the same importance ratio.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
Why distinguish GGPO behavior and reference policies?
Gibbs-Gap Policy Optimization for Off-Policy RL with Verifiable Rewards
Show answer and example
Answer: Behavior generates the sampled data; reference anchors KL. They needn't match. Refreshing reference changes the later objective without resampling old data.
Illustrative example: One frozen policy collects replies and another anchors deviation penalties.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Core mechanism
Why keep RKL everywhere and add FKL only at high disagreement?
Divergence-Routed On-Policy Distillation for Language Models
Show answer and example
Answer: RKL imitates covered modes but can weakly move near-absent teacher modes. JS routing localizes FKL's coverage correction. Union support, detached advantages and PPO matter; not switching all KL or only updating the sampled token.
Illustrative example: If teacher strongly prefers A and student only B, extra local FKL can lift A.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Implementation details
Should ties at DrOPD's kth-largest JS be broken to keep exactly K?
Divergence-Routed On-Policy Distillation for Language Models
Show answer and example
Answer: No. Retain all at threshold, possibly more than K. Compute K from valid positions as specified; K=0 adds no FKL. Deduplicate and renormalize the top-k teacher/student union. Freeze teacher, route and advantage.
Illustrative example: Don't drop tied boundary positions to force an exact quota.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Method comparison
How does DrOPD routing differ from replacing the whole loss with forward KL?
Divergence-Routed On-Policy Distillation for Language Models
Show answer and example
Answer: Add FKL only on routed positions and keep RKL globally. Full replacement changes concentration/coverage elsewhere. PPO candidate-advantage updates also aren't a static two-KL gradient.
Illustrative example: Low-disagreement format tokens keep the original objective; difficult positions get extra correction.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Basics
Why add forward KL at high-disagreement positions in DrOPD?
Divergence-Routed On-Policy Distillation for Language Models
Show answer and example
Answer: Reverse-KL gradients depend on student mass and can be weak on teacher modes students nearly ignore. Forward KL restores missing coverage locally while retaining RKL everywhere.
Illustrative example: Teacher-supported digits with near-zero student mass need an extra coverage signal.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Core mechanism
How can training-only physical time series improve a video-only encoder?
Time-JEPA: Grounding Video World Models with Physical Time Series
Show answer and example
Answer: Auxiliary heads predict speed or joint angles from video and send gradients back to the encoder. Remove the head at deployment; sensor values never enter inputs. Mask invalid frames and fit normalization on training only.
Illustrative example: Train with synchronized speed labels and test on video alone. Supplying speed at test would change the task.
Sections in the Chinese source: 背景与主要改动, 核心公式与隔离要求. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Implementation details
Can a fully missing variable in one tubelet be filled with a valid zero?
Time-JEPA: Grounding Video World Models with Physical Time Series
Show answer and example
Answer: No. Average valid frames independently per variable; empty tubelets remain invalid. Fit and freeze normalization on training. Apply SmoothL1 with β=0.5 only to valid entries, with domain-specific heads, statistics, and weights.
Illustrative example: If neither frame has a joint-angle measurement, skip that variable instead of treating missing as a true zero angle.
Sections in the Chinese source: 核心公式与隔离要求, 本地复现. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Method comparison
What does physical supervision add beyond masked latent prediction?
Time-JEPA: Grounding Video World Models with Physical Time Series
Show answer and example
Answer: Latent prediction encourages predictability without ensuring precise physical variables survive. Synchronized labels directly encourage those variables in the encoder. Benefits depend on variable coverage and distribution; one domain's probe does not establish universal physical or control ability.
Illustrative example: A model can predict road texture while ignoring speed differences until a speed target requires them.
Sections in the Chinese source: 背景与主要改动, 核心公式与隔离要求. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Basics
Why are action classes different from exact speeds or joint angles?
Time-JEPA: Grounding Video World Models with Physical Time Series
Show answer and example
Answer: Classification can be invariant across speeds and poses, while control and physical analysis need continuous detail. Time-JEPA adds temporal physical supervision without using sensors at deployment. Correct labels cannot establish numeric precision.
Illustrative example: Two clips can both show “raising a hand” with different speeds and elbow angles.
Sections in the Chinese source: 背景与主要改动, 核心公式与隔离要求. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Core mechanism
How do SMRC and PMRC recover gradient directions outside low-rank momentum?
Memory-Efficient Subspace Optimization with Residual Correction: From Momentum SGD to Muon
Show answer and example
Answer: Use low-dimensional momentum inside the subspace and current raw orthogonal residual outside, without persistent full residual state. SMRC uses low-dimensional EMA; PMRC modifies matrix direction treatment, and basis changes differ. Neither is just projected SGD.
Illustrative example: Keep momentum for two dominant directions while adding useful current gradient from a third.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Implementation details
Why correct SMRC parameters and rerun backward before refreshing the basis?
Memory-Efficient Subspace Optimization with Residual Correction: From Momentum SGD to Muon
Show answer and example
Answer: At stage end, correct parameters and zero momentum. A new forward/backward at the changed parameters supplies the refresh gradient. A stale gradient mismatches state; local β weights the new gradient, unlike some APIs.
Illustrative example: Do not reuse the gradient from before the parameter correction.
Sections in the Chinese source: 核心公式, 已实现的组件. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Method comparison
Why aren't SMRC and PMRC the same optimizer?
Memory-Efficient Subspace Optimization with Residual Correction: From Momentum SGD to Muon
Show answer and example
Answer: SMRC maps EMA momentum back directly; PMRC first applies matrix sign. SMRC corrects parameters at stage boundaries, unlike the theoretical PMRC shift rule. One projected SGD function cannot substitute.
Illustrative example: Applying msign to M before mapping generally changes direction.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Basics
Why not keep only the low-rank projected gradient?
Memory-Efficient Subspace Optimization with Residual Correction: From Momentum SGD to Muon
Show answer and example
Answer: Dominant singular directions can contain noise and exclude useful descent. Low-dimensional momentum handles in-subspace gradients; current orthogonal residual restores outside directions without full residual momentum.
Illustrative example: A smaller useful direction should not be permanently excluded by a large principal direction.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Structured Decisions / Decisions on Demand: Selective Reasoning, Transfer, and Feedback
Core mechanism
Why separate starting reasoning, triggering readout and accepting an answer?
Selective Reasoning for Typed Decision Models: When to Start, Read, and Stop
Show answer and example
Answer: The launch gate estimates benefit, the trigger chooses a natural prefix, and post-read normalized entropy decides acceptance/cancellation. They act on different states. Mark forced returns at budget exhaustion separately from threshold success; fitting cannot see evaluation gold.
Illustrative example: Example: Reason until a readable point, continue if the readout remains uncertain, and mark budget-limited uncertain output as forced rather than confident.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Structured Decisions / Decisions on Demand: Selective Reasoning, Transfer, and Feedback
Implementation details
Why normalize entropy by log K when candidate count changes?
Selective Reasoning for Typed Decision Models: When to Start, Read, and Stop
Show answer and example
Answer: Maximum entropy changes with K, so H/log K makes certainty thresholds comparable. Distinguish cancellation from budget exhaustion. Freeze gate normalization statistics and weights before evaluation, not by test feedback.
Illustrative example: Example: Four and ten choices have different raw entropy ranges; one raw threshold changes the stopping rule.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Structured Decisions / Decisions on Demand: Selective Reasoning, Transfer, and Feedback
Method comparison
How does reading at natural generation states differ from fixed-token polling?
Selective Reasoning for Typed Decision Models: When to Start, Read, and Stop
Show answer and example
Answer: The trigger seeks positions suitable for explicit readout; a rate limit only bounds frequency. A shared reader and entropy rule still decide reliability. Triggering is not proof of correctness.
Illustrative example: Example: Read after an intermediate conclusion rather than mid-equation, but still inspect candidate probabilities.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Structured Decisions / Decisions on Demand: Selective Reasoning, Transfer, and Feedback
Basics
Why not generate equally long reasoning for all Selective Reasoning questions?
Selective Reasoning for Typed Decision Models: When to Start, Read, and Stop
Show answer and example
Answer: Direct readout suffices for some choices; others need extra steps. Gate reasoning by expected benefit and read answers mid-generation to allocate compute where it is needed.
Illustrative example: Example: Return a simple intent immediately, but reason through multi-step arithmetic and stop when sufficiently certain.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Core mechanism
Why score true RoPE pairs with Q/K norm products rather than K magnitude alone?
Show answer and example
Answer: Attention error depends on both query and key. Score each rotation plane jointly to preserve coupling. Choose high-precision pairs per shared KV head during prefill and reuse them without future decode data. Memory adjacency need not be a true pair.
Illustrative example: A large key direction rarely used by queries may not deserve precision; adjacent Llama channels need not rotate together.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Implementation details
Can Llama/SmolLM2 RoPE pairs be formed from adjacent coordinates?
Show answer and example
Answer: No for split-half layout: pair corresponding coordinates across the halves, unlike interleaved layout. Pin the prefill-selected mask across pages, without decode lookahead.
Illustrative example: Pair index i in the first half with index i in the second, not necessarily i+1.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Method comparison
Are PN-Mix 1-bit correction and FP16 residual buffering the same?
Show answer and example
Answer: No. Correction shifts 1-bit reconstruction bins without extra residual codes. The buffer holds new not-yet-quantized tokens. They have distinct numerical and storage roles.
Illustrative example: Corrected quantized groups can coexist with a new full-precision token batch.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Basics
Why assign higher precision to some RoPE pairs?
Show answer and example
Answer: Rotation planes differ in QK impact. Prefill average Q/K norm products estimate importance, and each KV head fixes its high-precision set, trading cache width for attention error.
Illustrative example: Preserve both dimensions of a high-impact rotation pair together.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Core mechanism
How does OOB provide near-dataset-scale anti-collapse supervision with a small batch?
Dataset-Level Regularization for Joint-Embedding Self-Supervised Learning
Show answer and example
Answer: Periodically encode the training set and cache anchors per sample, then read fixed targets during small-batch optimization. This avoids estimating the full distribution every step but requires refresh as the model changes. It is not a minibatch variance or covariance loss.
Illustrative example: If memory holds eight images, precompute full-training-set quantile targets and match each of those eight to its own anchor.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Implementation details
How should refresh directions, targets, and gradients be isolated?
Dataset-Level Regularization for Joint-Embedding Self-Supervised Learning
Show answer and example
Answer: Freeze refreshed directions and per-sample targets, detach them and store on CPU. Normalize online loss by batch B, views V, and directions M; no gradients through refresh. The spherical variant optimizes unit anchors, not Gaussian targets. Stable sample IDs are required.
Illustrative example: After dataloader shuffling, look up each original ID rather than pairing the first cached anchor with the first current image.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界与成本结论. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Method comparison
How does Gaussian sorted-quantile matching differ from enforcing zero mean and unit variance?
Dataset-Level Regularization for Joint-Embedding Self-Supervised Learning
Show answer and example
Answer: Sorted quantiles assign normal targets to samples along random projections, constraining distributions and assignments. Mean and variance constrain only two moments. This is not the original Epps–Pulley statistic; matched baselines need the same quantile loss.
Illustrative example: Equal mass at −1 and +1 has mean zero and variance one but not normal quantiles.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Basics
Why might a small-batch SSL model align views without learning distinguishable features?
Dataset-Level Regularization for Joint-Embedding Self-Supervised Learning
Show answer and example
Answer: Positive alignment alone allows a constant encoder. Anti-collapse needs overall spread, which small batches estimate poorly. OOB uses broader statistics under memory constraints, but downstream performance still needs independent evaluation.
Illustrative example: Several similar embeddings may reflect genuinely similar images or global collapse; view agreement alone cannot tell.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Core mechanism
Why normalize current and related-task memory separately in Mosaics+?
Open-world AI: a Tale of Three Requirements
Show answer and example
Answer: One softmax lets numerous irrelevant past entries dilute current evidence. Two-stage retrieval normalizes each source then combines predictions. Short-term memory handles recent dependence, long-term stores predictive associations, and illegal future reads stay excluded.
Illustrative example: Three relevant current entries should not be diluted solely because there are 1,000 old records.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Implementation details
How does predictive value at i+1 avoid a future read?
Open-world AI: a Tale of Three Requirements
Show answer and example
Answer: Restrict i≤t−s with s≥1. Then i+1 is within the visible boundary. Split normalization at document boundaries and handle empty memory; reference loops and future perturbations are negative controls.
Illustrative example: With skip=1, i=t−1 may use t but not t+1.
Sections in the Chinese source: 核心公式, 结果与独立对照. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Method comparison
How does two-stage Mosaics+ differ from a single softmax pool?
Open-world AI: a Tale of Three Requirements
Show answer and example
Answer: Separate current and related-task normalization precedes combining. In a shared pool, adding even unrelated old entries can change current weight; these are different computations.
Illustrative example: Many old items should not drown out a few important current associations by count alone.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Basics
Why evaluate test-time learning rather than IID performance alone?
Open-world AI: a Tale of Three Requirements
Show answer and example
Answer: IID testing does not show use of newly observed task experience. Mosaics+ combines recent dependence and long-term predictive associations to organize new-task evidence.
Illustrative example: Use newly observed input/output examples to infer a rule rather than repeat old statistics.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Core mechanism
Why subtract decoding error on true next representations from decoding error on predicted ones?
One-Step Prediction Fidelity Is a Poor Proxy for Planning Performance in JEPA World Models
Show answer and example
Answer: Decoding error mixes encoder readability, decoder limitations, and prediction error. Using the same frozen decoder for both isolates excess error attributable to prediction more closely. It can be negative and must not be clipped. Long-horizon planning needs separate paired tests.
Illustrative example: If real representations decode two cells off and predicted ones three cells off, do not attribute all three cells to prediction.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Implementation details
Should negative E_excess be clipped to zero?
One-Step Prediction Fidelity Is a Poor Proxy for Planning Performance in JEPA World Models
Show answer and example
Answer: No. It is decoded error from predictions minus decoded error from true representations; predictions may happen to fit the frozen decoder better. Identical representations must yield zero. Fit state variance on shared training data, never refit the decoder on predicted test latents.
Illustrative example: True error three and predicted error two gives −1; clipping hides decoder extrapolation behavior.
Sections in the Chinese source: 核心公式, 本地复现, 聚合、结果与边界. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Method comparison
Why measure one-step physical error and planning success separately?
One-Step Prediction Fidelity Is a Poor Proxy for Planning Performance in JEPA World Models
Show answer and example
Answer: A planner repeatedly queries the model; errors can accumulate around key actions or alter search. One-step means miss closed-loop effects. Test actual planning with paired seeds and the same checkpoint and task.
Illustrative example: Better accuracy on common stationary states can lower average error even if near-collision mistakes make plans crash.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Basics
Why does a small one-step JEPA latent loss not establish understanding of physical motion?
One-Step Prediction Fidelity Is a Poor Proxy for Planning Performance in JEPA World Models
Show answer and example
Answer: Latents use encoder-specific coordinates; closeness need not mean accurate position, angle, speed, or long-horizon planning. This work checks physical decoding and actual planning separately, with shared training scales.
Illustrative example: Similar latent vectors may decode to different object angles, so measure physical error and planning success.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Core mechanism
Why can a sharp image with the right camera pose still fail PCC?
ESC: Escaping 2D Diffusion Ambiguity with Geometric Constraints for Consistent Novel View Synthesis
Show answer and example
Answer: Realistic appearance does not ensure the same 3D scene. PCC uses bidirectional depth reprojection and visibility to detect vanished or hallucinated objects, respecting occlusion and invalid pixels. Best-of-N selects geometric consistency, not the prettiest candidate.
Illustrative example: A cup can disappear in a visually clean view; bidirectional reprojection detects the inconsistency.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Implementation details
Why do reprojection checks need z-buffering and valid-pixel masks?
ESC: Escaping 2D Diffusion Ambiguity with Geometric Constraints for Consistent Novel View Synthesis
Show answer and example
Answer: Multiple source points can land on one pixel and must respect occlusion. Negative depth, out-of-bounds projections, and missing source depth are not valid errors. PCC also uses thresholds, logarithmic compression, clipping, and coverage normalization. No local implementation or capability metrics.
Illustrative example: A wall can correctly occlude an object. Lack of a visible match is not disappearance.
Sections in the Chinese source: 核心公式, 本地复现. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Method comparison
How does PCC Best-of-N differ from selecting the closest RGB image?
ESC: Escaping 2D Diffusion Ambiguity with Geometric Constraints for Consistent Novel View Synthesis
Show answer and example
Answer: Correct viewpoints can change pixels. PCC projects source and target depths both ways with occlusion and valid coverage checks, detects missing or hallucinated geometry, and minimizes the score. This does not optimize every image-quality metric.
Illustrative example: A duplicate source image can have the smallest RGB difference while failing the requested camera view.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Basics
Why does ESC care whether objects persist across generated viewpoints?
ESC: Escaping 2D Diffusion Ambiguity with Geometric Constraints for Consistent Novel View Synthesis
Show answer and example
Answer: Generators can render plausible posed images while adding or dropping objects. ESC links RGB and depth to test geometric consistency. New views should differ in pixels in spatially explainable ways, not merely look similar or good.
Illustrative example: A cup should move in projection as the camera circles a table, not vanish in an unoccluded region.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Core mechanism
Why learn latent actions instead of mapping human hand trajectories directly to robot joints?
EgoJEPA: Beyond Retargeting for Dexterous Manipulation from Egocentric Videos
Show answer and example
Answer: Human hands and robot actuators have different geometry. Motion-difference conditioning learns latent actions linked to visual changes, then joint modeling and robot-specific interfaces adapt them. What transfers is the effect, not coordinate equivalence.
Illustrative example: Fingers pinching and a gripper closing have different angles but can both lift a cup.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Implementation details
How does visibility of future visual targets differ between JEPA training and control?
EgoJEPA: Beyond Retargeting for Dexterous Manipulation from Egocentric Videos
Show answer and example
Answer: In training, an encoder supplies detached targets from future frames for prediction from history and hand-motion differences. Control cannot read an unrealized future. Local work is a literature/cost record with no adapter, metrics, or GPU receipt.
Illustrative example: Use the later object-position frame to supervise prediction, not as a deployment input.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Method comparison
How do action-conditioned JEPA and later joint visual/latent-action flow matching differ?
EgoJEPA: Beyond Retargeting for Dexterous Manipulation from Egocentric Videos
Show answer and example
Answer: First predict future visual representations conditioned on hand-motion differences, linking latent actions to observable effects. Then jointly model visual and action latents and adapt a robot I/O interface. Prediction and joint generation are separate and do not by themselves establish robot control.
Illustrative example: Learn which hand changes produce which visual effects before adapting the learned joint structure to an arm.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Basics
Why learn from human egocentric videos for transferable robot experience?
EgoJEPA: Beyond Retargeting for Dexterous Manipulation from Egocentric Videos
Show answer and example
Answer: Robot demonstrations are costly and embodiment-specific. Human video has action–appearance relationships but not robot control signals. Learn latent visual and action structure before adapting to a robot; actual control still needs environment validation.
Illustrative example: Human drawer-opening video teaches scene change, not direct robot-arm joint commands.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why are verbatim messages and omission markers more transparent than a fluent regenerated summary?
Show answer and example
Answer: Verbatim text preserves commitment wording. Omission markers identify what was moved out and provide a route back to the full conversation. This avoids silently turning guesses into facts. A recovery index does not mean the model found the evidence, and historical search can require paid subagent inference.
Illustrative example: Example: Summarizing 'do not deploy for now' without 'for now' changes the requirement. Keeping the original wording and marking removed tool logs makes the compression easier to audit.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
Why must Pi branch structure and tool metadata survive even when body text is removed?
Show answer and example
Answer: Read only ancestors of the chosen leaf so sibling histories cannot leak in. Tool pairing IDs, names, errors, and exit states preserve invocation semantics after body removal. A character cap evicts the oldest full blocks and leaves recovery markers, so the active context no longer contains every original statement.
Illustrative example: Example: Preserve isError=false rather than treating false as missing; otherwise later compression can lose proof the call succeeded.
Sections in the Chinese source: 核心公式, 本地复现, 代码映射与边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
Why can't deterministic compression and model-subagent context_lookup both be called zero-call?
Show answer and example
Answer: Compression rules need no model, but semantic historical search uses a model subagent and has separate capability and cost. The local run restores exact IDs and checks hashes, not the paper's model retrieval. Fewer visible characters indicate a view change, not memory-QA ability or proportional total storage savings.
Illustrative example: Example: Reading a log by its known ID is model-free. Finding which commitment matters without an ID may need extra inference.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Why are user commitments and constraints especially easy to lose when compressing long coding sessions?
Show answer and example
Answer: Large tool logs and discussions compete for context. Summaries can omit small but binding phrases or turn conditional requests into unconditional ones. Transparent compression marks what stays and what moves, with a path back to original materials, so continuation knows what is visible and what needs retrieval.
Illustrative example: Example: 'Submit only after confirming tests' summarized as 'prepare to submit' can make a later agent mistake conditional permission for unconditional authorization.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Core mechanism
Why shouldn't response latency and action age be combined into one metric?
The World Won't Wait: Evaluating and Teaching Agents Under a Live Clock
Show answer and example
Answer: Response latency ends when the model returns; action age also includes queuing and applying the action. A live environment keeps changing, so a fast response can still be obsolete when applied. A slow teacher may practice and refine guidance from feedback only between attempts, and independent validation decides whether to replace it. It cannot peek at future live-test states to correct actions.
Illustrative example: The model decides to jump within one second, but the command is queued for another two seconds and the obstacle is already gone. The three-second action age explains the failure better than one-second response latency alone.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Implementation details
Does TORSI accept new guidance on a validation tie, and what was tested locally?
The World Won't Wait: Evaluating and Teaching Agents Under a Live Clock
Show answer and example
Answer: No. Complete pairs with identical task IDs must show a strictly positive mean gain, and feedback, validation, and test remain separate. Timestamps must be finite and ordered. The local tests cover timing and numeric selection contracts only, with no game engine, Jev, or teacher run. Fixture differences in action age are not improvements in game progress.
Illustrative example: If old and new guidance earn the same validation score, do not replace the old one because the new explanation is longer. Nor can you select it retroactively because its test score was slightly higher.
Sections in the Chinese source: 核心公式, 本地复现, 成本与边界. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Method comparison
How does slow-teacher online takeover differ from TORSI's offline guidance between attempts?
The World Won't Wait: Evaluating and Teaching Agents Under a Live Clock
Show answer and example
Answer: Online takeover places the slow model in real-time control, where delay can make its decisions obsolete. TORSI has the slow teacher analyze feedback and practice between attempts; guidance is then frozen and the fast student executes independently. The teacher cannot revise from test data. Guidance is accepted only for strictly positive gains on independent validation, and replay practice is not live execution.
Illustrative example: A teacher can explain a failed tracking run offline. In the next live run, the student must choose actions quickly on its own rather than pausing the world for step-by-step answers.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Basics
Why might a real-time agent reason better but still act incorrectly because it responds too slowly?
The World Won't Wait: Evaluating and Teaching Agents Under a Live Clock
Show answer and example
Answer: The environment keeps changing during inference, so an action based on an old observation can be stale by execution. TORSI studies improving a small execution policy under real-time constraints without putting all high-quality guidance on the online critical path. Account for offline learning and online response separately; static accuracy is not effective action in a live environment.
Illustrative example: A model spends a long time choosing a correct action for a moving target's old position and misses when it acts. Both response speed and observation freshness matter.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why separate forming a composite capability from admitting it to a persistent skill library?
Recursive Formation and First-Class Reuse of Compositional Capabilities
Show answer and example
Answer: Compatible interfaces show that operations compose, not that the composition meets intended behavior. Independent validation gates admission. Persistent identity binds parents, types, and composition, while loading checks references and acyclicity. Later direct invocation saves search, not the real underlying computation.
Illustrative example: Example: Sorting then deduplicating can be wired together but may fail on empty input. Validate before storing it for direct reuse.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
Why recheck the canonical hash, parent references, and acyclicity on cold load?
Recursive Formation and First-Class Reuse of Compositional Capabilities
Show answer and example
Answer: Persistent identity comes from canonical JSON of types, operations, and parent IDs. Files can be corrupted or tampered with. Complete parent references and acyclicity ensure legal recursion. True persistence testing destroys the old object and cold-loads the same snapshot. A local sequence-domain result is not general Lean or LLM capability.
Illustrative example: Example: A file can set its own ID as a parent without changing its name, creating a cycle. Keeping a live object in memory does not verify recovery from disk.
Sections in the Chinese source: 核心公式, 本地复现, 代码映射、验证与边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
What work does invoking an admitted composite save over expanding and searching again?
Recursive Formation and First-Class Reuse of Compositional Capabilities
Show answer and example
Answer: A first-class composite can participate as a typed operation with an identity, so later tasks do not repeat formation search. Expanding again must recover parent operations and wiring. Reuse saves search and assembly, while inner computation still runs. Count calls, formation attempts, and underlying work separately.
Illustrative example: Example: Reuse a learned filter-then-sort operation directly. Filtering and sorting still execute; skipping assembly does not make them free.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
What distinguishes a first-class composite capability from a stored formation history?
Recursive Formation and First-Class Reuse of Compositional Capabilities
Show answer and example
Answer: A first-class capability has its own type, public call ID, and execution semantics, so it is a candidate in later compositions. Formation history records how it was assembled and may require searching again. Direct reuse avoids formation work, but still runs recursive computation on the current input rather than caching an old answer.
Illustrative example: Example: Append 01 can become an ordinary operator by ID. It still appends both items to today's input, not returning yesterday's result.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Core mechanism
Why can't public execution materials be relabeled T_full to recompute the full report gap?
What Coding Agents Leave Unsaid: Measuring Task-State Gaps Between Execution and Reports
Show answer and example
Answer: The original full view includes some unpublished reasoning or memory evidence. Without it, observer capability and answerable questions change. Pair probes and weight important facts, but separate public and full conditions. Cluster by task/run rather than assuming many probes for one task are independent.
Illustrative example: If a port appears only in private memory, a public-log observer's failure cannot represent the original full-view ceiling.
Sections in the Chinese source: 背景与主要改动, 核心公式与统计单位. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Implementation details
Why resample CTXGAP intervals by task/run rather than individual probes?
What Coding Agents Leave Unsaid: Measuring Task-State Gaps Between Execution and Reports
Show answer and example
Answer: Probes within one task share execution and reporting conditions and are not independent. Cluster while keeping paired conditions together. Preserve category weights and equal task aggregation rather than replacing the main gap with pooled accuracy differences.
Illustrative example: A task with 100 similar probes is not 100 independent tasks and should not artificially narrow confidence intervals.
Sections in the Chinese source: 核心公式与统计单位. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Method comparison
How does CTXGAP importance-weighted accuracy differ from averaging all probes equally?
What Coding Agents Leave Unsaid: Measuring Task-State Gaps Between Execution and Reports
Show answer and example
Answer: More probes in one category should not outweigh another category's important facts. Divide the number of important atomic facts by probes in the category, sum weighted scores, and normalize. Equal weighting can let many easy or similar probes hide omissions.
Illustrative example: Many questions about a trivial log should not obscure one missing acceptance fact.
Sections in the Chinese source: 背景与主要改动, 核心公式与统计单位. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Basics
How does the CTXGAP reporting gap differ from task completion rate?
What Coding Agents Leave Unsaid: Measuring Task-State Gaps Between Execution and Reports
Show answer and example
Answer: It compares cumulative-report and full-execution observers on the same probes, measuring how well users can recover true task state. A completed task can omit key facts in its report; a detailed report does not establish success.
Illustrative example: Code may be fixed while the report omits an unrun check, leaving users unable to judge acceptance accurately.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Core mechanism
Why freeze the original retrieved skill set before meaning-preserving rewrites?
Evolving Together, Failing Apart: The Skill-Policy Co-evolution Trap in Self-Improving Agents
Show answer and example
Answer: Retrieval could change which skills are delivered, confounding wording sensitivity with ranking. Freeze selected identities and retrieval fields, then change only content. String similarity does not prove semantic equivalence, and unchanged aggregate success can hide bidirectional flips.
Illustrative example: Provide the same three skills after paraphrasing. If three different skills are retrieved, outcome changes no longer isolate wording.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Implementation details
Is the number of no-skill successes the CODA RR denominator?
Evolving Together, Failing Apart: The Skill-Policy Co-evolution Trap in Self-Improving Agents
Show answer and example
Answer: No. RR is the proportion of all matched tasks successful under both no-skill and perturbed-skill conditions. Conditioning on successes changes the definition. Keep identical ID sets and reject missing tasks.
Illustrative example: Of 100 tasks, 20 no-skill successes and 10 joint successes give RR=10%, not 50%.
Sections in the Chinese source: 核心公式, 本地复现. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Method comparison
Why does CODA report per-task CR alongside aggregate success?
Evolving Together, Failing Apart: The Skill-Policy Co-evolution Trap in Self-Improving Agents
Show answer and example
Answer: Success totals can remain unchanged when some successes become failures and other failures become successes. CR measures agreement across conditions per task; repeated unperturbed runs estimate stochastic noise.
Illustrative example: If success and failure swap across two tasks, total success remains 50% but CR is zero.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Basics
Why does CODA freeze retrieval before rewriting skills?
Evolving Together, Failing Apart: The Skill-Policy Co-evolution Trap in Self-Improving Agents
Show answer and example
Answer: It audits dependence on delivered content and wording. Reretrieval changes selection and ranking, so fix retrieval-side fields and replace content only.
Illustrative example: If a rewritten skill is no longer retrieved, wording dependence cannot be distinguished from recall changes.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Core mechanism
Why may a repaired trajectory propose a pending tool call but not write a successful tool result?
ARIA: Runtime Inspection and Amendment for Multi-Agent Systems
Show answer and example
Answer: Results must come from execution; otherwise the patch fabricates evidence. The cut preserves the observed prefix, and continuation proposes new actions before comparison with a fresh control. Structural validity does not prove correct attribution or repair.
Illustrative example: Example: Proposing to rerun tests is legal. Saying all tests passed in the continuation is not; execute and observe them.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Implementation details
What hard constraints apply to the cut event, patch types, and size?
ARIA: Runtime Inspection and Amendment for Multi-Agent Systems
Show answer and example
Answer: The cut retains the referenced observed event and, if nonempty, must precede the identified bad event. A patch allows at most eight events and 12,000 canonical JSON characters, requires nonconflicting IDs, and cannot replay the old suffix with new IDs. Only assistant messages and pending calls are allowed, not fabricated tool results. No five-stage model training ran locally.
Illustrative example: Example: Propose another order lookup, but do not insert an invented tool observation that it was paid.
Sections in the Chinese source: 核心公式, 本地复现, 成本与边界. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Method comparison
Why require a patched pass and fresh-control failure together for verified repair?
ARIA: Runtime Inspection and Amendment for Multi-Agent Systems
Show answer and example
Answer: A patched success alone can result from retry randomness or environmental changes. If the fresh control also succeeds, improvement cannot be attributed to a necessary patch. The pair rules out simple retries as repair evidence but is not exact causal replay of a full framework. The paper reconstructs continuation context within a stated scope.
Illustrative example: Example: Tests pass after patching, but also pass on a fresh unpatched rerun. Do not use that pair as verified repair training data.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Basics
Why repair an early partial trajectory rather than just summarize failure at the end?
ARIA: Runtime Inspection and Amendment for Multi-Agent Systems
Show answer and example
Answer: Early mistakes propagate across agent messages. Cutting and continuing during a run can prevent amplification. ARIA learns when and where to intervene and how to continue, but must execute the patch to verify it. Replacing a suffix does not prove diagnosis or exact environment rollback.
Illustrative example: Example: A message uses the wrong unit and later agents copy it. Repair before propagation rather than only explain it after completion.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Core mechanism
Why do retries add attempts without adding logical samples or choosing the best result?
AlphaDiana: A Measurement System for Agents in Verifiable Reasoning Tasks
Show answer and example
Answer: Retries of a task/sample address the same request. Changing denominators or selecting the best attempt distorts sampling and hides failure costs. Report the latest record and sum costs over attempts; separate execution and grading. Ungraded is unknown, not correct or silently excluded.
Illustrative example: If a task times out then succeeds, charge both attempts' tokens while retaining one sample. Do not keep only the cheap success.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Implementation details
When must AlphaDiana return unknown for complete tokens per correct result?
AlphaDiana: A Measurement System for Agents in Verifiable Reasoning Tasks
Show answer and example
Answer: Return None if there are no confirmed correct samples, any attempt lacks token counts, or any requested sample lacks execution records. Report known tokens and coverage separately; partial measurement is not complete cost.
Illustrative example: If one of two attempts has no token count, even an eventual correct result cannot establish full cost per correct sample.
Sections in the Chinese source: 核心公式, 无 gold 边界与限制. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Method comparison
What evaluation difference arises from selecting the latest rather than best retry?
AlphaDiana: A Measurement System for Agents in Verifiable Reasoning Tasks
Show answer and example
Answer: Retries leave the logical denominator unchanged, and the protocol reports latest status. Best-of selection introduces an implicit extra-attempt advantage. Costs accumulate across every attempt, not only the one shown.
Illustrative example: If a success is followed by a failed retry, latest-status reporting cannot restore the earlier success and charge only one execution.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Basics
Why does AlphaDiana separate execution status from grading status?
AlphaDiana: A Measurement System for Agents in Verifiable Reasoning Tasks
Show answer and example
Answer: A successful run may lack a valid grader decision. Interrupted grading is unknown, neither correct nor incorrect. Separate statuses and full planned denominators expose execution failures, missing grades, and coverage.
Illustrative example: If a tool completes but the grader times out, mark ungraded rather than removing the sample.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Core mechanism
Why does repeated-layer LoOPSD cost compute and need validation route selection?
LoOPSD: On-Policy Self-Distillation with Looped Teachers
Show answer and example
Answer: Loops change distribution without new weights but execute more layers per token and search multiple routes. Select validation improvements over standard route, then distill student prefixes; never choose on test. More compute doesn't guarantee gains.
Illustrative example: Rereading a chapter may help but takes time; choose using practice tasks, not final exam answers.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Implementation details
Do teacher weights/routes get gradients during LoOPSD distillation?
LoOPSD: On-Policy Self-Distillation with Looped Teachers
Show answer and example
Answer: Freeze both for the round; update standard student via RKL. Training references can be used only for permitted route search, not student rollout. Count repeated-layer compute.
Illustrative example: Five repeated layers are not free even without new parameters.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Method comparison
How does re-searching loops each round differ from a fixed initial loop?
LoOPSD: On-Policy Self-Distillation with Looped Teachers
Show answer and example
Answer: Student updates can change useful repeat regions. Search adapts, and continue only if independent validation beats standard routing, not test-selected routes.
Illustrative example: A useful repeated middle region can stop helping after the student learns it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Basics
Why create LoOPSD teachers by repeating layers rather than using bigger weights?
LoOPSD: On-Policy Self-Distillation with Looped Teachers
Show answer and example
Answer: Shared parameters can compute a better distribution through a longer loop. Search contiguous repeat regions and distill on student prefixes. Extra compute, not external answers, supplies guidance.
Illustrative example: Repeating middle layers changes per-token computation without adding parameters.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Core mechanism
Why fix true state and run a reference plan to prove solvability before generating conflict scenarios?
Show answer and example
Answer: Otherwise failures may stem from contradictions or an unusable interface rather than peer-conflict handling. Executable references establish true state, permitted goals, and obligations, then evaluation tests whether the agent finishes from the environment. Empty requirements or missing verdicts are not automatic success, and fairness adjustments must be preregistered.
Illustrative example: A peer may claim a file was deleted while it still exists and the reference can complete the task. This tests whether the agent checks the environment rather than facing an impossible case.
Sections in the Chinese source: 背景与主要改动, 核心公式与公平性口径. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Implementation details
Why shouldn't CLASHarness match records by list position when downgrading checks for fairness?
Show answer and example
Answer: Identify checks by original task_id/check_index, mark specified required items diagnostic, and union overlapping categories. Reordering or duplicate labels can cause wrong removals or double counts if based on positions.
Illustrative example: A check in two disputed categories should appear only once in the downgrade set.
Sections in the Chinese source: 核心公式与公平性口径. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Method comparison
Why report CLASHarness complete success separately from partial check pass rate?
Show answer and example
Answer: Complete success requires a verified nonempty required list with every item passed. Partial rate reports how many checks passed. An empty conjunction or incomplete verdict list does not establish full success.
Illustrative example: Passing three of four required checks gives 75% partial credit and no complete success.
Sections in the Chinese source: 核心公式与公平性口径. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Basics
Why does CLASHarness execute a reference plan before agents handle conflicting messages?
Show answer and example
Answer: Reference execution verifies solvability under fixed true state and compiles checks from observed effects. Otherwise failure may reflect a broken scenario, not inability to handle false facts, stale state, or conflicting peer instructions.
Illustrative example: If a peer claims a file changed, verify actual state and a reachable goal first.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Core mechanism
Why aren't source-present and source-free scores comparable by task ID alone?
AutoformBench: A Lean Formalization Benchmark for Advanced Mathematics
Show answer and example
Answer: Visible materials change difficulty: a source or proof offers definitions and strategies, while a source-free workspace requires more autonomous construction. Matching task names do not align information. All obligations must pass, and failures, budget overruns, and infrastructure errors stay in the denominator.
Illustrative example: For the same theorem, one group reads a textbook proof while another gets only a Lean skeleton. Match source conditions before attributing score differences to the model.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Implementation details
Can the AutoformBench interface drop failed, over-budget, or infrastructure-error rows?
AutoformBench: A Lean Formalization Benchmark for Advanced Mathematics
Show answer and example
Answer: No. Keep the full fixed task set and actual CSV costs. Nominal caps do not imply equal spend. Importing existing verdicts is not running Harbor; mark planned runs as such if unexecuted.
Illustrative example: Counting only successfully launched tasks hides environment failures and inflates results.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Method comparison
How does AutoformBench differ from filling one isolated Lean proof with all definitions given?
AutoformBench: A Lean Formalization Benchmark for Advanced Mathematics
Show answer and example
Answer: It also requires constructing definitions and supporting lemmas, then satisfying every obligation. An isolated proof usually starts with more context fixed. Shared Lean verification does not mean equal autonomous formalization work or comparable pooled success.
Illustrative example: If a definition is missing, formalize it and prove supporting lemmas. Filling only the final by block is incomplete.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Basics
Why does AutoformBench require all proof obligations in a task to pass?
AutoformBench: A Lean Formalization Benchmark for Advanced Mathematics
Show answer and example
Answer: Advanced formalization requires definitions, auxiliary lemmas, and final proof to work together. A single fragment is insufficient. Trusted Lean/Mathlib must verify all obligations; partial counts do not replace task success.
Illustrative example: A main theorem with a dependency still containing sorry is not a complete task.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why expand both DHPO reward matching and policy gradients gradually?
WeGR: Unified Generative Retrieval for Industrial Search Systems in WeChat
Show answer and example
Answer: Short matches densify initially sparse rewards and restrict updates to evaluated tokens. Expand to full codes over time. Multi-positive SFT and CoCo-VAE serve separate roles; local evidence is bounded.
Illustrative example: Reward outdoor first, then camping, then the full tent path without reinforcing wrong unevaluated suffixes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Does WeGR refresh low-utilization codes at every layer?
WeGR: Unified Generative Retrieval for Industrial Search Systems in WeChat
Show answer and example
Answer: Refresh from first-layer hit EMA only. Keep deeper layers fixed for refresh, align DHPO prefix length to schedule, and add multisession-positive NLL per session. Full-sequence loss is not progressive horizon.
Illustrative example: At h=1, later codes must not receive full gradients from the prefix reward.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
How does CoCo-VAE collision loss differ from entropy?
WeGR: Unified Generative Retrieval for Industrial Search Systems in WeChat
Show answer and example
Answer: Collision measures overlap across examples; entropy measures one distribution's spread. Uniform distributions can have high entropy and completely overlap.
Illustrative example: Assigning every item uniformly across all codes is not discriminative.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Why grow WeGR preference optimization from short SID prefixes?
WeGR: Unified Generative Retrieval for Industrial Search Systems in WeChat
Show answer and example
Answer: Full paths yield sparse reward; coarse prefixes provide early signal. Align rewarded horizon with optimized tokens to avoid updating unevaluated suffixes.
Illustrative example: Learn broad category before requiring an exact item code.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why predict target side information but mask semantic-to-prediction reads?
UniSGR-V2: Stepwise Target-Side Feature Anticipation and Crossing for Generative Recommendation
Show answer and example
Answer: Target features are unavailable before decoding and estimates help each level. Prevent noisy estimates from propagating into earlier semantic states, but retain output conditioning. Ranking supervision anchors targets. Full training is not reproduced locally.
Illustrative example: Price/style estimates can guide output while staying out of prior semantic attention.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Why not replace the real auxiliary attribute tower with self-predictions?
UniSGR-V2: Stepwise Target-Side Feature Anticipation and Crossing for Generative Recommendation
Show answer and example
Answer: A learnable target needs business labels; mutual prediction can collapse. Isolate future labels and use type-specific masks so semantic states cannot read true future attributes.
Illustrative example: Two networks can agree on a constant without learning attributes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
How does a one-way mask differ from disabling predicted features?
UniSGR-V2: Stepwise Target-Side Feature Anticipation and Crossing for Generative Recommendation
Show answer and example
Answer: Semantic tokens cannot attend to predicted tokens; prediction can read prior semantics, and output conditioning still uses predicted features. Disabling removes the mechanism.
Illustrative example: Prevent hidden noise feedback without withholding estimated features from generation heads.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Why predict side features for an item not generated yet?
UniSGR-V2: Stepwise Target-Side Feature Anticipation and Crossing for Generative Recommendation
Show answer and example
Answer: Item identity is unknown mid-decode. Predict continuous features by level and cross them with generation/ranking; auxiliary tasks prevent target collapse.
Illustrative example: Estimate likely attributes before emitting a brand-related code.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
Why shrink estimated skill gains by reliability before setting skill-enhanced teacher distillation weights?
Show answer and example
Answer: Some skills are useless or worse than the student, so equal trust can transfer regressions. Independent controls and same-start replay estimate utility differences. Shrink noisy estimates toward the group mean and use more reliable gains to adjust distillation trust. Skill reward and shrinkage-based trust have distinct roles; invalid replay is not paired evidence.
Illustrative example: One lucky successful trial merits shrinkage because the gain is uncertain. A skill that works repeatedly in same-start controls can receive more distillation weight.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
Does the local project have an RSIPO adapter, real skill replay, and policy training?
Show answer and example
Answer: No. It records the specification and cost decision only, with no paper adapter, metrics, or GPU pass. An implementation must preserve current-policy skills, valid same-start replay, independent controls, and training/test separation. Another skill operator or fixed rule cannot substitute for full RSIPO or Evolve integration.
Illustrative example: A working GRPO trainer does not establish skill utility estimation or reliability-weighted OPSD. Without those, the algorithm is not implemented.
Sections in the Chinese source: 核心公式, 本地复现, 边界与后续触发条件. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
How does dynamic skill–policy co-updating differ from a fixed offline skill library in freshness?
Show answer and example
Answer: A fixed library stabilizes retrieval conditions but preserves experience from old policies and tasks. Co-updating builds skills from current trajectories and tests benefits with independent controls and same-start replay, tracking changing ability. It adds nonstationarity and evaluation cost. Update counts are not benefit or evidence of a reliable teacher.
Illustrative example: After a policy learns a shorter evidence path, a new skill can reflect it. A fixed old library may continue recommending the obsolete path.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why does RSIPO update old skills as the policy improves?
Show answer and example
Answer: Older skills reflect older policies and can become redundant, obsolete, or worse than direct solving by the current student. RSIPO forms skills from current trajectories and tests same-start gains, allowing skills and policy to co-evolve. Assistance is not inherently beneficial; keep independent controls and do not include original answers in evaluation input.
Illustrative example: An early skill uses a long search while the current policy finds evidence directly. Re-measure instead of forcing an older, potentially slower skill.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Core mechanism
Why distinguish MicroRel and MacroRel and vet criteria first?
LLM-based User-Product Relevance Evaluation for Dynamic Product Advertising
Show answer and example
Answer: Compressed history supplies intent; a criteria-trained detector judges candidates. Pair-level and user-level averages answer different questions, but neither repairs wrong relevance judgments. Private review and commercial model workflows are not reproduced.
Illustrative example: With 100 and ten candidates, user one weighs ten times as much in micro but equally in macro.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Implementation details
Can gold relevance labels stand in for detector predictions?
LLM-based User-Product Relevance Evaluation for Dynamic Product Advertising
Show answer and example
Answer: No. Train a real detector on discovered, human-reviewed business criteria and evaluate independently. Rule discovery has bootstrap and per-class F1 requirements; coverage review cannot be skipped.
Illustrative example: Copying gold computes a score without learning relevance.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Method comparison
Why can MicroRel and MacroRel disagree?
LLM-based User-Product Relevance Evaluation for Dynamic Product Advertising
Show answer and example
Answer: Micro averages user–candidate pairs, while macro weights users equally. Users with many candidates dominate micro more. Report both rather than cherry-pick.
Illustrative example: Active users with many candidates affect micro disproportionately.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Basics
Why compress intent rather than equate clicks with Snap DPA relevance?
LLM-based User-Product Relevance Evaluation for Dynamic Product Advertising
Show answer and example
Answer: Clicks are sparse and exposure-biased. Group recent behavior by SID and event type, then apply business-aligned relevance criteria. This is a separate demand metric, not renamed CTR.
Illustrative example: Repeated recent cart additions can be stronger intent evidence than one exposure.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Structured Decisions / Candidate Invariance: Order, Competitors, and Calibration
Core mechanism
Why audit added questions, evidence and candidates separately?
The Company a Question Keeps: Auditing Open Jev-Style Models for Retrieval
Show answer and example
Answer: Adding a question keeps proposition and evidence fixed to test interference. New evidence legitimately changes context; new candidates change normalization. Combining them prevents attribution. For expansion, compare original conditional odds. Evaluator labels, scenario names and document identity never enter predictions.
Illustrative example: Example: An irrelevant extra question flipping a decision is concerning; new evidence may justify a flip, and a third choice can reduce shares without changing relative preference.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Structured Decisions / Candidate Invariance: Order, Competitors, and Calibration
Implementation details
Why is a binary probability of exactly 0.5 not a strict flip?
The Company a Question Keeps: Auditing Open Jev-Style Models for Retrieval
Show answer and example
Answer: Strict flips require centered probabilities on opposite sides, with product strictly negative. Equality is a separate tie. Assigning the boundary arbitrarily overstates flips and obscures precision effects.
Illustrative example: Example: 0.5 followed by a preference for yes is a boundary change, not a clear no-to-yes reversal.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Structured Decisions / Candidate Invariance: Order, Competitors, and Calibration
Method comparison
How can candidate-expansion probability drops be distinguished from preference reversals?
The Company a Question Keeps: Auditing Open Jev-Style Models for Retrieval
Show answer and example
Answer: A new candidate enlarges the softmax denominator and can reduce old shares naturally. Compare conditional odds between original choices or strict binary flips to separate normalization from relative preference.
Illustrative example: Example: A loses share but still outranks B because C absorbs mass; A/B order has not flipped.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Structured Decisions / Candidate Invariance: Order, Competitors, and Calibration
Basics
Why is request independence separate from structured output?
The Company a Question Keeps: Auditing Open Jev-Style Models for Retrieval
Show answer and example
Answer: Valid JSON or candidate labels guarantee format, not whether an unrelated question changes the original decision. The audit fixes evidence and target to measure interference from grouping requests.
Illustrative example: Example: Both questions can produce valid probabilities even if placing them together flips the first decision.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Core mechanism
Why isn't min(functional success rate, safety success rate) the joint success rate?
Harmony Harness: Multi-Objective Self-Evolution of Agent Harnesses
Show answer and example
Answer: The two marginal rates can come from different trajectories. Joint success requires both criteria to be met in the same execution. Multiobjective search uses preferences and neighborhood replacement to maintain trade-offs rather than increasing functional scores by weakening safety. Final selection and held-out reporting sets also need to remain separate, or reporting environments become search feedback.
Illustrative example: In ten executions, eight can be functionally successful and eight safe while their failures do not overlap. Joint success can then be only six, so taking the minimum marginal count of eight overstates it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Implementation details
Can the final max-min selection choose the best harness on held-out OOD tasks?
Harmony Harness: Multi-Objective Self-Evolution of Agent Harnesses
Show answer and example
Answer: No. Select the final harness by max-min over functionality and safety estimates from the search set, leaving held-out ID and OOD for evaluation. Joint success requires both criteria on the same trajectory. Locally there is no MOEA/D, real Docker environment, or exploit evaluation; a scalarization formula does not validate complete self-evolution.
Illustrative example: If the marginal rates are 80% functional and 70% safe, joint success is not necessarily 70%. Failures may occur on different tasks, so intersect successes trajectory by trajectory.
Sections in the Chinese source: 核心公式, 本地复现与成本判断. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Method comparison
How does weighted Chebyshev differ from fixed linear weighting when retaining candidates?
Harmony Harness: Multi-Objective Self-Evolution of Agent Harnesses
Show answer and example
Answer: Chebyshev uses the largest weighted deviation from the ideal point, emphasizing the weakest objective. Different preference vectors maintain separate harnesses; replacements require strict improvement in the neighborhood, with at most two per step. A fixed linear sum can let a large gain on one objective offset losses on another. The selection geometries differ, but both depend on reliable evaluation.
Illustrative example: For a candidate with 90% functional success and 30% safety, maximum deviation highlights the safety weakness, whereas a simple average may still beat a more balanced option.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Basics
Why not combine functionality and safety into an unconstrained average first?
Harmony Harness: Multi-Objective Self-Evolution of Agent Harnesses
Show answer and example
Answer: Better functionality might come from relaxed checks, and an average can let a high functionality score offset serious safety regression. Harmony keeps both objectives and searches through preference subproblems and neighborhood replacement so deployments account for each. It is not a proof of safety against all attacks, and separate marginal rates are not joint success on the same trajectory.
Illustrative example: A trajectory can finish the task while leaking protected data. Mark functional success and safety failure rather than averaging it into a passing result.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Core mechanism
Why require joint perturbation, retraining, and consumer review when each feature seems unnecessary alone?
Show answer and example
Answer: Redundant features compensate during one-at-a-time tests. Check weak dependence over time and across heads, then joint removal and retraining, followed by every downstream consumer. One neutral run is not deletion authority.
Illustrative example: Age and age buckets can substitute for each other, yet deleting both can hurt new-user tasks.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Implementation details
Why can signed_change be negative, and how does VirtualBatch isolate frozen dependence?
Show answer and example
Answer: Higher-is-better perturbation minus baseline is negative for damage. Reuse dense/sparse/ragged inputs and perturbation maps while switching references; parameter digests rule out updates. Joint effects cannot be summed from individual effects.
Illustrative example: AUC 0.80→0.79 shows harm, not benefit; training during the probe would change its meaning.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Method comparison
What do frozen perturbation and retraining answer separately?
Show answer and example
Answer: Perturbation measures current reliance; retraining tests adaptation to absence. Neither covers all online effects. Redundancy can hide reliance, and retraining does not erase a failed time gate. Local three-seed baseline retention is a valid negative.
Illustrative example: Age removal can hurt a frozen model but recover after retraining; joint removal can hurt despite harmless single deletion.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Basics
Why is feature retirement a lifecycle task rather than an importance ranking?
Show answer and example
Answer: Features may feed multiple heads and systems, and training can restore reliance. Audit time stability, joint deletion, retraining, online evidence, and lineage before retiring all consumers.
Illustrative example: The main click head may not use a field that fraud models still require.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Core mechanism
Why learn paragraph cuts rather than fine-tune UpTurn's whole reasoner?
Inject a Turn, Not a Token: Learning Where to Redirect LLM Reasoning for Self-Correction
Show answer and example
Answer: Freezing isolates when to insert a correction; rewards come from actual regenerated suffixes at different cuts. Keep prefix, close turn, insert fixed user turn, and obey leftover budget. It isn't whole rewrite or model training. Gate cannot use gold.
Illustrative example: Retry from a paragraph boundary with less remaining budget, not unlimited free retries.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Implementation details
Why not give UpTurn the full L_max again after cutting?
Inject a Turn, Not a Token: Learning Where to Redirect LLM Reasoning for Self-Correction
Show answer and example
Answer: Prefix already consumes budget; suffix gets L_max minus cut position. No parseable answer means gate zero, threshold fixed. Some mechanism seeds still fail; execution is not stable success.
Illustrative example: Full budget after a late cut gives treatment extra generation cost.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Method comparison
How does a new dialogue turn differ from appending reflection to the assistant message?
Inject a Turn, Not a Token: Learning Where to Redirect LLM Reasoning for Self-Correction
Show answer and example
Answer: Role and conditioning boundaries change. End the assistant turn, add a new user instruction and continue within remaining budget. String append doesn't imply equivalent chat semantics.
Illustrative example: A user's request to recheck differs from the assistant continuing to reflect.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Basics
Does UpTurn learn token edits or where to restart correction?
Inject a Turn, Not a Token: Learning Where to Redirect LLM Reasoning for Self-Correction
Show answer and example
Answer: It learns paragraph cut and repair gate with frozen reasoner/encoder. Keep the prefix, insert a fixed new user turn and continue with the reasoner.
Illustrative example: Restart after a bad derivation section instead of changing old token probabilities.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
Why stop gradient through TAPO's prefix geometric ratio?
Trajectory-Aware Policy Optimization: Closing the State Distribution Gap in Long-Horizon LLM RL
Show answer and example
Answer: The ratio measures state drift, but current credit should not backpropagate into all earlier tokens. Freeze the geometric weight and preserve the current log-probability gradient separately. This avoids cross-position gradients and length dilution.
Illustrative example: Adjust step ten for prefix drift without rewriting credit for steps one through nine.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
How does TAPO prevent prefix weighting from dividing current-token gradients by position?
Trajectory-Aware Policy Optimization: Closing the State Distribution Gap in Long-Horizon LLM RL
Show answer and example
Answer: Detach the geometric ratio and multiply by current probability divided by its detached copy. Forward keeps the weight; backward follows only current action, without prior gradients or inverse-prefix dilution.
Illustrative example: Step ten can use ten-step context without automatically receiving one-tenth gradient.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
How does prefix geometric mean differ from multiplying probability ratios?
Trajectory-Aware Policy Optimization: Closing the State Distribution Gap in Long-Horizon LLM RL
Show answer and example
Answer: Exponentiate average log ratio to limit extreme length scaling. This is a different estimator, not an equivalent numerically stable product.
Illustrative example: Many small per-step shifts can create an extreme product while the geometric mean retains typical scale.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
Why track prefix-state shift rather than only the current ratio in trajectory TAPO?
Trajectory-Aware Policy Optimization: Closing the State Distribution Gap in Long-Horizon LLM RL
Show answer and example
Answer: Small earlier shifts accumulate into later visitation changes. A current action ratio misses prior branching. Prefix geometric log-ratios supply trajectory context with tighter position-dependent bounds.
Illustrative example: A similar final-action ratio does not mean earlier branches left its state unchanged.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Core mechanism
Why rewrite with a larger related model and check the answer again?
Continually Post-training Small Models via Synthetic Self-distillation
Show answer and example
Answer: It can adapt training demonstrations to students but may introduce mistakes. Accept through answer checking and test fidelity/distribution requirements. Do not use evaluation answers to rewrite or condition the student.
Illustrative example: Verify that a simplified solution reaches the same conclusion; using test solutions invalidates generalization.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Implementation details
Where may SynthSelf use gold?
Continually Post-training Small Models via Synthetic Self-distillation
Show answer and example
Answer: Offline training demonstrations can guide and validate rewrites. Never give evaluation answers to student or use them for curriculum. Real SFT remains necessary; preserve failed and repaired evidence separately.
Illustrative example: Text filtering without a parameter update isn't complete training.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Method comparison
How does answer rejection sampling differ from validating format?
Continually Post-training Small Models via Synthetic Self-distillation
Show answer and example
Answer: Valid structure does not ensure correctness. Check rewritten answers, then test distribution/fidelity constraints empirically. Same family guarantees neither.
Illustrative example: A syntactically correct JSON result with wrong arithmetic should be rejected.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Basics
Why use a larger same-family SynthSelf rewriter?
Continually Post-training Small Models via Synthetic Self-distillation
Show answer and example
Answer: Small self-rewrites can distort content and original style may be difficult for students. A larger aligned rewriter supplies candidates verified by answer checks; shorter isn't automatically better.
Illustrative example: Convert a proof with skipped steps into learnable steps while retaining the correct result.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Core mechanism
Why omit old reasoning for the teacher and not execute extra actions?
State-OPD: State-Grounded On-Policy Distillation for Multi-Turn Agents
Show answer and example
Answer: Executed observations are real even when student reasoning is mistaken. Use them for teacher state while student retains full context. Compare oversampled candidates at the same high-entropy state; executing them breaks alignment. Entropy is vocabulary-wide.
Illustrative example: Use actual click results instead of mistaken prior narrative, and score extra proposed clicks without pressing every button.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Implementation details
Can sampled-token surprisal replace State-OPD entropy?
State-OPD: State-Grounded On-Policy Distillation for Multi-Turn Agents
Show answer and example
Answer: No. Average full-vocabulary entropies across tokens and follow the stated state selection/count formula. Keep sampled-K1 double clipping rather than heuristic teacher ranking.
Illustrative example: A sampled rare word does not imply high entropy of the whole distribution.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Method comparison
How does high-entropy oversampling differ from executing all extra actions?
State-OPD: State-Grounded On-Policy Distillation for Multi-Turn Agents
Show answer and example
Answer: Generate extra reasoning/actions for training comparison, but execute none of them. The actual environment follows the original trace. Executing them changes state visitation and budget.
Illustrative example: Scoring alternative tools doesn't mean using every one.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Basics
Why hide old reasoning from the State-OPD teacher?
State-OPD: State-Grounded On-Policy Distillation for Multi-Turn Agents
Show answer and example
Answer: Old reasoning can be wrong. The teacher sees executed actions/observations to avoid inheriting false narratives; student still sees its full history.
Illustrative example: Teacher should trust a tool's failure result over previous reasoning claiming success.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Core mechanism
Why ignore an answer found in the nondeclared evidence stream?
SOAR: Source-Aware Reinforcement Learning with Precise Credit Assignment for Search Agents
Show answer and example
Answer: A declared source should actually support the answer; cross-stream rescue would reward invented dependency. Separate recall/retrieval, keep old reasoning and external evidence out of recall, and exclude observations from actor actions.
Illustrative example: If a reply cites retrieval but only recall contains the fact, provenance is inconsistent.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Implementation details
Can SOAR award matching credit from the other evidence stream?
SOAR: Source-Aware Reinforcement Learning with Precise Credit Assignment for Search Agents
Show answer and example
Answer: No. Check only the declared stream. Tool observations and recall text aren't actor-loss actions. Explicitly normalize reward before group advantage; mechanism execution isn't external capability.
Illustrative example: If an answer claims recall but only a webpage supports it, correctness doesn't earn source credit.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Method comparison
Why can't SOAR internal recall see prior reasoning or retrieval?
SOAR: Source-Aware Reinforcement Learning with Precise Credit Assignment for Search Agents
Show answer and example
Answer: The audit distinguishes existing knowledge from external findings. Isolate recall before external evidence; contamination invalidates source matching.
Illustrative example: Recalling a fact after reading a webpage isn't independent memory.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Basics
Why require explicit source claims in SOAR answers?
SOAR: Source-Aware Reinforcement Learning with Precise Credit Assignment for Search Agents
Show answer and example
Answer: Correctness alone doesn't verify evidential dependence. Record internal recall and external retrieval separately, then test whether the claimed stream supports the answer.
Illustrative example: A retrieval claim must be supported by retrieval records, not by a lucky memory.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Core mechanism
Why use unrouted teachers and impose a target gradient norm floor?
Beyond Domain-Routed Supervision: Cross-Teacher Guarding for Multi-Teacher On-Policy Distillation
Show answer and example
Answer: Auxiliary teachers check cross-domain harm on identical states. Gate token credit, then address real gradient conflicts without overweakening target learning. Norm floors are tradeoffs, not universal gains; arbitrary vectors don't replace objectives.
Illustrative example: Code guidance can protect prefixes during math updates, but full removal might stall math learning.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Implementation details
Why not remove the entire conflicting component in Road projection?
Beyond Domain-Routed Supervision: Cross-Teacher Guarding for Multi-Teacher On-Policy Distillation
Show answer and example
Answer: Choose maximum feasible projection while preserving the target norm floor; full projection can violate it. Compute actual auxiliary gradients, not static scores or registry labels.
Illustrative example: Near antiparallel gradients can make a fully projected update vanish.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Method comparison
Can Road token gating replace gradient projection?
Beyond Domain-Routed Supervision: Cross-Teacher Guarding for Multi-Teacher On-Policy Distillation
Show answer and example
Answer: No. Gating scales local supervision; projection handles shared-parameter conflict with a target norm floor. A weight formula isn't the complete mechanism.
Illustrative example: Compatible scores do not rule out gradient conflict.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Basics
Why query unrouted teachers in Road-MOPD?
Beyond Domain-Routed Supervision: Cross-Teacher Guarding for Multi-Teacher On-Policy Distillation
Show answer and example
Answer: Shared parameters can degrade other domains even if examples route separately. Evaluate other teachers on the same prefixes for compatibility and real auxiliary gradients.
Illustrative example: Math updates can affect code behavior despite separate data routing.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Core mechanism
Why profile teacher branch uncertainty, then select by student loss?
Show answer and example
Answer: Monte Carlo continuations identify fragile points on correct traces; shrink by teacher profile, then probe students and train on current high-loss items. Rerouting follows learning. Reduced student exposure is not end-to-end speedup.
Illustrative example: Keep a fragile proof first and switch after mastery; correctness alone cannot choose.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Implementation details
Why doesn't reduced exposure imply equivalent total GPU acceleration?
Show answer and example
Answer: Exposure counts student training only, omitting real teacher profiling, probes and overhead. Log separately and keep evidence of generation.
Illustrative example: Expensive screening may outweigh two fewer training epochs.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Method comparison
What do teacher profiling and student NLL rerouting each solve?
Show answer and example
Answer: Teacher stage finds potentially difficult reasoning states; student probes find current gaps. Teacher difficulty isn't permanent student difficulty and isn't length.
Illustrative example: Move budget if the student already mastered a teacher-profiled difficult step.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Basics
Why select high branching uncertainty among correct trajectories?
Show answer and example
Answer: Correct endings can hide fragile branch points. Monte Carlo continuations identify teachable states beyond binary original correctness.
Illustrative example: A proof with confusing mid-course formula options is useful despite a correct ending.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Core mechanism
Why remeasure success instead of using fixed question length?
Show answer and example
Answer: Difficulty is policy- and budget-dependent, so real-rollout success bands shift with learning. Select phases on validation and retain hard-stage controls; length reward doesn't replace difficulty or justify test selection.
Illustrative example: A short arithmetic problem may be harder for current model than a familiar long problem.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Implementation details
May phase selection use test scores or length reward to define difficulty?
Show answer and example
Answer: No. Select by validation, restore selected checkpoint and reestimate pass rates from actual current rollouts. Length reward is auxiliary, not difficulty. Static mock rates aren't online.
Illustrative example: Long output alone does not make a problem hard.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Method comparison
Why isn't online fixed-medium equivalent to progressing to hardest band?
Show answer and example
Answer: Fixed-medium preserves learnable success rates. Hardest progression can lose signal, and reported hardest-stage regression should remain visible.
Illustrative example: A curriculum crossing capability may yield only failures rather than progress.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Basics
Why does GRPO curriculum difficulty change as policy improves?
Show answer and example
Answer: Measure difficulty through current sampled pass rate; learning and forgetting move items between bands. Static text length or old scores aren't the same.
Illustrative example: Yesterday's failed short prompt can become easy today.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
Why can the same f-divergence give different trust regions with MC, atomic or binary estimation?
Show answer and example
Answer: Topology decides how action and complement probabilities enter. Even fixed function/radius need not yield the same ratio interval. GPPO separates these choices and uses the resulting bounds. Preserve different KL forms instead of renaming GRPO.
Illustrative example: Constraining only a rare action versus also the remaining mass can allow different changes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
Why expose literal KL and two-sided log separately in GPPO?
Show answer and example
Answer: Some algorithm mappings and printed formulas differ; keep both explicit rather than silently fix one. Ratios must be positive and binary complements valid; solve the selected trust region instead of copying PPO bounds.
Illustrative example: A configuration called KL that actually computes two-sided log obscures what was verified.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
Why are MC, atomic and binary topologies more than implementation details?
Show answer and example
Answer: They count sampled actions and unsampled complements differently, changing estimated divergence and admissible ratios. Name mappings establish no equivalence without the matching definitions.
Illustrative example: Binary accounts for one minus action probability, not just a renamed atomic result.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
What two independent choices define GPPO's proximal updates?
Show answer and example
Answer: Choose f-divergence geometry and its estimation topology. Combinations allow different ratio ranges; a common framework does not make instances train identically.
Illustrative example: Different treatment of complement probability can change trust regions for the same divergence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Core mechanism
Why align decoded spans before projecting teacher log probabilities?
CAST: Credit Alignment for Cross-Tokenizer On-Policy Distillation
Show answer and example
Answer: Position-wise IDs lack a common event. Verified text spans provide corresponding regions, then uncertainty/smoothness allocate conserved credit. Anchor unmatched text and disclose alignment gaps.
Illustrative example: If one teacher token becomes several student tokens, verify identical decoded span before allocating its score.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Implementation details
How does CAST uncertainty weighting preserve chunk credit?
CAST: Credit Alignment for Cross-Tokenizer On-Policy Distillation
Show answer and example
Answer: Normalize weights from uncertainty and local smoothness, then multiply chunk total advantage. Anchor unmatched positions to initial student rather than drop them and claim full coverage.
Illustrative example: A higher token share must not increase total chunk advantage.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Method comparison
How does CAST allocation differ from CSD shared events?
CAST: Credit Alignment for Cross-Tokenizer On-Policy Distillation
Show answer and example
Answer: CAST distributes total teacher span credit over student positions. CSD builds common-event probability distributions with REST. They conserve different objects.
Illustrative example: Conserving one span's advantage is not preserving all event probability mass.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Basics
Why align decoded spans for cross-tokenizer CAST?
CAST: Credit Alignment for Cross-Tokenizer On-Policy Distillation
Show answer and example
Answer: Token IDs and boundaries differ. Match equivalent decoded spans, then project teacher span logp into student positions.
Illustrative example: Two Chinese tokenizations may split a word differently but decode to the same span.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Core mechanism
Why count shared prefix only once?
Branching On-Policy Distillation
Show answer and example
Answer: Branching widens continuations, not earlier supervision. Select uncertain/disagreeing forks and sample independent frozen-student suffixes for teacher scores, without teacher forcing or gold filters.
Illustrative example: Count three shared steps once, then each of two suffixes separately.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Implementation details
Why not count shared BranchOPD prefix loss for every branch?
Branching On-Policy Distillation
Show answer and example
Answer: Added coverage is in suffixes. Deduplicate prefix and align valid positions. PBR supervises defined endings; boxed-then-EOS isn't arbitrary early stop.
Illustrative example: Five branches shouldn't multiply unrelated early-token learning fivefold.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Method comparison
How does disagreement branching differ from retaining only correct branches?
Branching On-Policy Distillation
Show answer and example
Answer: It doesn't filter by outcome; success-only changes distribution after sampling. Preserve teacher scoring, termination supervision and matched cost/provenance.
Illustrative example: Failed branches still expose visited states; don't remove them for prettier scores.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Basics
Why sample extra suffixes at uncertain BranchOPD forks?
Branching On-Policy Distillation
Show answer and example
Answer: One rollout shows only one continuation. Frozen student branches expose multiple choices at the same state, focusing teacher supervision on disagreement rather than teacher-selected text.
Illustrative example: Try two continuations independently from the same proof prefix.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Core mechanism
Why choose capability-boundary tasks then regenerate and verify teacher traces?
Show answer and example
Answer: Value depends on current expert–student gap and old tasks need rechecking. Generate and verify anew to avoid carrying stale wrong answers. Priming and verified supervision have different rules.
Illustrative example: Retire mastered decompositions and validate the teacher's new boundary solution instead of reusing an unchecked draft.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Implementation details
What must BoundaryEvo record before SFT?
Show answer and example
Answer: Select tasks from current student/fixed expert/analyzer gap and quality, generate fresh verified teacher traces, and supervise only response tokens. No test references or old answers mislabeled fresh.
Illustrative example: A previous answer for another question breaks selection-to-supervision alignment.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Method comparison
How does capability priming differ from verified teacher supervision?
Show answer and example
Answer: Priming is small-scale behavioral warmup without correctness filtering. Later select questions, generate fresh teacher responses and independently validate. Don't present warmup as verified.
Illustrative example: Style imitation and checking a solution's correctness use different acceptance rules.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Basics
Why reevaluate old BoundaryEvo problems each round?
Show answer and example
Answer: Student capability changes; mastered boundary questions should leave replay and new ones enter. Curriculum coevolves rather than replay static hard problems.
Illustrative example: Move to new gaps after mastering an old proof.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Core mechanism
Why does CV-OPD give both teachers the same student prefix and change only the visual crop?
CV-OPD: Contrastive Vision On-policy Distillation
Show answer and example
Answer: Differences then primarily come from the region, not text history. On its own rollout prefixes, the student moves toward the correct-region teacher and away from the negative-region teacher by a bounded JSD margin. This addresses answer distillation that might learn only a language prior.
Illustrative example: For cup color, the positive crop shows the cup and the same-size negative crop shows background. Both teachers see identical student-generated text so far.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Implementation details
If at least one rollout is correct, should the other rollouts be dropped?
CV-OPD: Contrastive Vision On-policy Distillation
Show answer and example
Answer: No. Keep all rollouts of a sample that passes the at-least-one-correct gate. DAG gates tokens across the retained batch with JSD strictly above the mean. Teachers, generation, and gate do not backpropagate. The top-100 JSD approximation must preserve the full tail bucket.
Illustrative example: If one of eight rollouts is correct, keep all eight; removing the other seven changes the training distribution.
Sections in the Chinese source: 核心公式, 本地复现, 本地结果与边界. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Method comparison
What does CDL add over distillation from a positive-crop teacher alone?
CV-OPD: Contrastive Vision On-policy Distillation
Show answer and example
Answer: Positive-only distillation pulls the student toward the positive distribution. CDL also requires sufficient JSD from the negative teacher using a margin on a bounded divergence. It neither requires unlimited separation nor guarantees improvements on every benchmark.
Illustrative example: A student equally close to both teachers may have low positive-only loss, but still incur a margin penalty for failing to distinguish regions.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Basics
Why prepare positive and negative crops from the same image in CV-OPD?
CV-OPD: Contrastive Vision On-policy Distillation
Show answer and example
Answer: The positive provides local evidence; the same-image, same-size, low-overlap negative is a visual counterfactual. The student sees the full image, while teacher inputs differ only in region. This tests whether learning follows the evidence rather than a language prior.
Illustrative example: For a street sign, use the sign in the positive crop and nearby wall in the negative crop. Both teachers share parameters.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Core mechanism
Why preserve original token positions and rectified-flow direction when shrinking action queries?
SmolWAM: A Latent World-Action Model for Robotic Manipulation on Edge Devices
Show answer and example
Answer: The pretrained trunk expects particular position semantics and a data-to-noise velocity. A lighter interface that changes them adds distribution shift. SmolWAM restores compatibility stage by stage; checkpoint loading alone does not establish equivalent behavior.
Illustrative example: Reading teacher velocity noise-minus-data with an opposite-sign recovery rule can move actions the wrong way.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Implementation details
Does SmolWAM use the same rectified-flow velocity direction as LeWAM?
SmolWAM: A Latent World-Action Model for Robotic Manipulation on Edge Devices
Show answer and example
Answer: No. SmolWAM follows teacher data-to-noise interpolation xτ=(1−τ)x0+τε, with velocity ε−x0 and a consistent clean estimate. LeWAM uses the opposite direction. Retain the 12 specified blocks, original positions, and zero-bias sink rather than borrowing another formula.
Illustrative example: Reversing velocity during recovery moves the action in the wrong direction even if loss code runs.
Sections in the Chinese source: 核心公式, 本地复现, 数据协议、结果与边界. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Method comparison
How does a single-frame VAE plus adapter differ from directly replacing the VAE?
SmolWAM: A Latent World-Action Model for Robotic Manipulation on Edge Devices
Show answer and example
Answer: The teacher expects a video-clip latent distribution. The adapter maps single-frame outputs back into that space to preserve a usable trunk interface. Matching shape or token positions alone does not align numeric scale or spatiotemporal semantics.
Illustrative example: Two encoders can output identically shaped tensors but different distributions; the adapter learns compatibility.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Basics
Why lighten the entire video-generative robot interface in SmolWAM?
SmolWAM: A Latent World-Action Model for Robotic Manipulation on Edge Devices
Show answer and example
Answer: Vision encoding, video backbone, text conditioning, and action representation all cost compute. A small action head alone can leave most costs. Each change needs compatibility checks and recovery training before real control validation.
Illustrative example: A tiny action head still pays for a large VAE, long visual input, and multistep denoising.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Core mechanism
Why commit only accepted inputs and keep a high-precision anchor separate from low-precision reads?
ReplaySSM: Cache SSM Inputs, Not State
Show answer and example
Answer: Replay outputs from a checkpoint and short input log without full writes every token. Rejected drafts cannot enter committed state. Refresh from the high-precision anchor and raw records to avoid rounding feedback. Reassociation preserves recurrence dependencies.
Illustrative example: Accept two of four drafts and advance only two steps; do not refresh from the rejected tail or rounded copy.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Implementation details
Why preserve raw values and recompute residuals when refreshing GDN?
ReplaySSM: Cache SSM Inputs, Not State
Show answer and example
Answer: Residuals depend on the precise anchor. Reusing rounded reads or stale residuals feeds error back. Refresh from committed raw records and separately produce high/low copies; zero accepted tokens leave the anchor unchanged.
Illustrative example: Rejected drafts may have computed outputs without updating committed recurrence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Method comparison
How should output-only replay match stepwise recurrence?
ReplaySSM: Cache SSM Inputs, Not State
Show answer and example
Answer: Compute equivalent readout without materializing each state. Persistent refresh uses only committed inputs. Computation and writes change, not the reference accepted sequence.
Illustrative example: Match ordinary recurrence on accepted tokens; rejected drafts affect neither committed state.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Basics
Why cache small inputs instead of writing a full matrix each token?
ReplaySSM: Cache SSM Inputs, Not State
Show answer and example
Answer: Matrix-state traffic can be bandwidth-bound. Refresh periodically and record inputs for reassociated reads, especially output-only speculative checks.
Illustrative example: Store one anchor and several small inputs instead of a whole matrix per step.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Core mechanism
How can Orthros decode from all history after local prefill?
Orthros: Phase-Aware Heterogeneous Attention for Efficient Transformers
Show answer and example
Answer: Base tokens use sliding attention but build full per-layer KV. Output-specific queries read shared full history where predictions are required and do not write KV. Fixed-window prefill compute is linear, not linear-attention state or reduced cache.
Illustrative example: During prefill base tokens read neighbors, while the final output query and future decode positions can retrieve full stored history.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Implementation details
Should Orthros's output stream append its own K/V?
Orthros: Phase-Aware Heterogeneous Attention for Efficient Transformers
Show answer and example
Answer: No. Same-layer KV comes only from base. Output queries use a shared projection and stream-specific affine transforms. Writing output KV changes the contract and may break cached-versus-batch parity.
Illustrative example: Append one base KV per step, not one per stream.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Method comparison
Is Orthros linear prefill kernelized linear attention?
Orthros: Phase-Aware Heterogeneous Attention for Efficient Transformers
Show answer and example
Answer: No. Fixed-window SWA costs O(Tw). The output stream still performs ordinary causal cross-attention over full KV; it neither compresses into recurrence nor discards old cache.
Illustrative example: KV at the start remains available even if most prompt positions never read it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Basics
Why separate base and output-specific tokens?
Orthros: Phase-Aware Heterogeneous Attention for Efficient Transformers
Show answer and example
Answer: Most prefill positions build cache rather than emit outputs. Base uses a fixed window and retains full KV; output-specific queries read it when predictions are needed.
Illustrative example: Most prompt positions in a long document need not perform global output reads.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Core mechanism
Why group QK contribution dimensions and calibrate OAL against output error?
Not All Dimensions Are Equal: Outlier-Aware Attention Linearization with Grouped Quadratic Kernels
Show answer and example
Answer: A few coordinates can dominate softmax approximation error. OAL finds stable post-RoPE groups and learns positive-definite quadratic kernels for shared groups and pairs. Vectors after aggregation, not only weights, reveal impact; positivity protects normalization.
Illustrative example: Equal weight errors have different effects if one location has a large or very different value.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Implementation details
Which OAL grouping, calibration, and initialization choices may use test data?
Not All Dimensions Are Equal: Outlier-Aware Attention Linearization with Grouped Quadratic Kernels
Show answer and example
Answer: None. Train supports grouping, calibration, and LoRA; validation selects initialization and checkpoint; test comes last. Keep ε in the positive kernel to ensure a positive denominator.
Illustrative example: Do not replace the validation-selected kernel with one found better on test and report that same test again.
Sections in the Chinese source: 核心公式, 数据协议. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Method comparison
How does OAL output MSE differ from fitting attention weights?
Not All Dimensions Are Equal: Outlier-Aware Attention Linearization with Grouped Quadratic Kernels
Show answer and example
Answer: Output depends on values and normalized aggregation, so equal weight errors can affect outputs differently. Calibrate the kernel to actual readouts.
Illustrative example: Swapping weights on identical values changes nothing, unlike swapping opposite values.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Basics
Why focus on high-contribution post-RoPE QK coordinates?
Not All Dimensions Are Equal: Outlier-Aware Attention Linearization with Grouped Quadratic Kernels
Show answer and example
Answer: Coordinate contributions are uneven; a high-contribution tail can dominate error. Grouping and positive quadratic kernels target these differences rather than treating every coordinate alike.
Illustrative example: A few coordinates driving most score changes merit different approximation from low-contribution ones.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Core mechanism
How do MSAR streams differ from channel-split heads, and where does softmax apply?
Multi-Stream Attention Residuals
Show answer and example
Answer: Each stream keeps the full hidden vector at a different sequence scale. Softmax retrieves depth sources separately per stream; sigmoid combines retrieved streams without unit-sum weights. Persistent expanded views also change cost.
Illustrative example: Raw, short, and long views each contain full vectors, then independently choose historical layers.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Implementation details
Is a zero source harmless before a block has a partial sum?
Multi-Stream Attention Residuals
Show answer and example
Answer: No. Zero still enters the depth softmax denominator and dilutes other sources. Omit the absent partial, and use sigmoid rather than stream softmax in the final mixer.
Illustrative example: A zero vector added to two valid sources changes their normalized weights.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Method comparison
How does Block MSAR's persistent summary differ from caching all expanded views?
Multi-Stream Attention Residuals
Show answer and example
Answer: Store embeddings, completed block summaries, and the current partial sum only. Expand streams at read time; permanently storing them changes capacity and lifetime.
Illustrative example: Four views can serve one read without retaining four historical copies.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Basics
Why retrieve earlier depth representations at the current layer?
Multi-Stream Attention Residuals
Show answer and example
Answer: Ordinary residuals accumulate updates in one vector. MSAR makes full-width views at several time scales and retrieves across depth to access history differently.
Illustrative example: Use raw, short-window, and smoother views to form a layer input.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Structured Decisions / Decisions on Demand: Selective Reasoning, Transfer, and Feedback
Core mechanism
Why can a well-calibrated model still route errors poorly by low confidence?
Show answer and example
Answer: Calibration checks probabilities against aggregate correctness; routing needs errors concentrated at lower confidence than correct cases. Select fraction on validation macro-F1 and freeze for test. Parallel languages share sentence identities; cluster comparisons by sentence rather than counting independent duplicates.
Illustrative example: Example: Mean confidence matches accuracy, but errors are spread across confidence levels. Routing the lowest-confidence cases may not save compute.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Structured Decisions / Decisions on Demand: Selective Reasoning, Transfer, and Feedback
Implementation details
What data may choose cascade fraction and local temperature?
Show answer and example
Answer: Validation predictions only: select fraction by the specified grid and macro-F1, and temperature by validation NLL. Freeze before test. Temperature calibration is a local extension, not the authors' training step.
Illustrative example: Example: If test suggests escalating more, don't revise the fraction and report the same score as independent.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Structured Decisions / Decisions on Demand: Selective Reasoning, Transfer, and Feedback
Method comparison
How does routing low-confidence examples differ from translating all examples first?
Show answer and example
Answer: A cascade allocates extra compute to selected cases; translation changes language representation for every input. Translation gains do not show error-detection skill. Select and freeze routing fraction on validation.
Illustrative example: Example: Escalating only difficult examples need not translate them. Translating everything into Indonesian changes even easy inputs.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Structured Decisions / Decisions on Demand: Selective Reasoning, Transfer, and Feedback
Basics
Why report accuracy, calibration and risk–coverage in regional-language transfer?
Show answer and example
Answer: Accuracy measures aggregate correctness; calibration matches predicted probabilities to frequencies; risk–coverage asks whether confidence filtering retains reliable examples. Each informs whether a low-cost model and stronger fallback make sense.
Illustrative example: Example: Models with the same accuracy may differ greatly in how concentrated errors are at low confidence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Core mechanism
Why train narrow-to-wide as well as narrow and wide paths?
EV-JEPA: Elastic V-JEPA for Any Device and Cross-Device Inference
Show answer and example
Answer: Individually strong widths need not exchange intermediate states successfully. The mixed route trains the widening boundary under a common frozen teacher. This differs from arbitrary slicing or splicing independent models and does not eliminate communication cost.
Illustrative example: A server's wide layers may not understand a phone's narrow intermediate state without handoff training.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Implementation details
Does EV-JEPA's teacher stay frozen or track the elastic student by EMA?
EV-JEPA: Elastic V-JEPA for Any Device and Cross-Device Inference
Show answer and example
Answer: Frozen V-JEPA 2 supplies common targets for narrow, wide, and mixed student paths. Nested student parameters share masked-token L1 supervision in one space. Channel slicing alone is not joint training or cross-device verification.
Illustrative example: Updating shared student weights leaves the teacher's outputs fixed, not updated through EMA.
Sections in the Chinese source: 核心公式, 本地复现, 边界与后续触发条件. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Method comparison
How does a learned split-layer compressor differ from dropping tokens?
EV-JEPA: Elastic V-JEPA for Any Device and Cross-Device Inference
Show answer and example
Answer: Dropping tokens may permanently hide evidence. A compressor trains reconstruction so transmitted representations remain useful to the second stage. Lower reconstruction loss is not end-to-end losslessness or proof of deployed handoff.
Illustrative example: Dropping half the spatial positions differs from encoding into fewer tokens and learning to reconstruct recoverable structure.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Basics
Why support phone, server, and split-device budgets for one video model?
EV-JEPA: Elastic V-JEPA for Any Device and Cross-Device Inference
Show answer and example
Answer: A fixed width may not fit all devices. Separate models may use incompatible representations. EV-JEPA learns shared elastic widths and compatible handoffs, but not every split is valid or free. Transfer, compression, and task quality need testing.
Illustrative example: Matching tensor shapes is not enough for a phone's narrow front half to hand off to a wide server back half.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Core mechanism
Why fit both row and column scales without calibration activations?
DualQuant: Dual-scale Calibration-free Weight-and-Activation Quantization for LLMs
Show answer and example
Answer: Row scaling cannot resolve shared outlier columns. Fit both to static weight reconstruction and use column scales for runtime activation rescaling. With fixed codes solve scalar least squares, project to legal scales, and requantize; code changes need not be monotone.
Illustrative example: A column large across many rows needs joint treatment and compatible input scaling.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Implementation details
Can column fitting use old codes after changing row inverse scales?
DualQuant: Dual-scale Calibration-free Weight-and-Activation Quantization for LLMs
Show answer and example
Answer: No. Project row inverse scales, synchronize inner copies, and requantize before solving columns, then requantize again. The supported INT4 subset is not MXFP4.
Illustrative example: Fitting columns to stale codes violates alternating optimization.
Sections in the Chinese source: 核心公式, 本地复现. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Method comparison
How does alternating closed-form scaling differ from heuristic exponent search?
DualQuant: Dual-scale Calibration-free Weight-and-Activation Quantization for LLMs
Show answer and example
Answer: Separate grid assignment and scales; solve scalar least squares at fixed codes, project to legal formats, and requantize. Heuristics do not implement this optimization or its conditional descent result.
Illustrative example: A fixed-code optimal scale must be reconsidered after codes change.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Basics
How can DualQuant rescale activations without calibrating on them?
DualQuant: Dual-scale Calibration-free Weight-and-Activation Quantization for LLMs
Show answer and example
Answer: Fit row/block and column scales from W alone. Apply learned column scales reciprocally to X at runtime. A solver that never reads X does not mean inference never transforms X.
Illustrative example: Derive column scales offline, then rescale input channels before quantization.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why are DoubleCat two-level interpolation and SemiCat permanent reveal distinct?
On the Design Space of Continuous Diffusion Language Models
Show answer and example
Answer: Interpolation shifts training toward interior simplex states to reduce endpoint mismatch. Permanent reveal leaves chosen tokens clean and fixed, supervises unrevealed positions, and adjusts hazard to remaining time. Neither adds default self-conditioning.
Illustrative example: Keeping a confirmed subject clean differs from training on interior probability patterns.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
How test final hazard one together with permanent carryover?
On the Design Space of Continuous Diffusion Language Models
Show answer and example
Answer: At the last step reveal all remaining positions. Preserve already revealed inputs and outputs at every step. Use untempered probabilities for continuous Euler and temperature only for categorical commits.
Illustrative example: A token revealed at step two cannot change at step three.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Method comparison
Does two-level interpolation require an extra denoiser call for self-conditioning?
On the Design Space of Continuous Diffusion Language Models
Show answer and example
Answer: No. Interpolation constructs inputs without another denoiser. SemiCat also needs no default extra call. Optional bootstrap self-conditioning is a separately costed ablation.
Illustrative example: Do not hide a second forward under a nominal one-call training budget.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Basics
Why reveal some tokens as clean one-hot vectors early?
On the Design Space of Continuous Diffusion Language Models
Show answer and example
Answer: High-dimensional Gaussian noise can swamp a norm-one signal. Revealed anchors remain fixed, while loss covers unrevealed positions.
Illustrative example: Confirmed words stay clean as other positions denoise.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why does ReadRaw only accept metadata IDs that have already been observed?
Show answer and example
Answer: Lightweight metadata provides traceable entry points. Reading raw text must bind to IDs actually returned, preventing fabricated links or access to unauthorized records. Memory builds for the current question through alternating search, reading, and answering. The metadata index is not fully understood memory and must not carry the final answer.
Illustrative example: Example: Search returns meeting M7, so ReadRaw can open its conversation. A model-invented M999 is not valid historical evidence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
What state is managed by excluding seen entries in SearchMeta versus appending to the query cache?
Show answer and example
Answer: Seen entries record metadata IDs already returned and prevent duplicate results in this question. The cache accumulates useful material obtained for reasoning and answering. Raw conversations remain traceable external sources; the cache is not a prefilled answer. Separate states let you check whether search gains information and trace answer evidence.
Illustrative example: Example: Once meeting A's entry is seen, search should seek other events. Keep A's decision in the cache after reading it; skipping its entry does not erase evidence already read.
Sections in the Chinese source: 核心公式, 本地复现, 结果、验证与边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
How does test-time memory construction differ from one retrieval with fixed context?
Show answer and example
Answer: One retrieval fixes evidence before later gaps are known. Test-time construction updates memory alongside information needs and can search or read more. This grants tool autonomy but can also waste steps; evaluate whether it improves real answers, not just action execution.
Illustrative example: Example: A meeting mentions rescheduling. Search for a subsequent event to confirm the new date instead of clinging to the date retrieved first.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Why store only lightweight metadata offline and build detailed memory at query time?
Show answer and example
Answer: What matters in a conversation depends on the question, and an offline summary can discard details too early. Event metadata retains lookup clues and raw links. SearchMeta, ReadRaw, and Answer build a query-specific cache at inference. Multiple accesses still cost work; this is not zero memory processing.
Illustrative example: Example: To answer an old travel-date question, find the event index and then read the source conversation rather than prewriting one universal summary.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Core mechanism
Why check policy–predictor pair compatibility after each component passes its own checks?
RSI-Anything: Unlocking Recursive Self-Improvement with World Models
Show answer and example
Answer: The policy may depend on the old predictor's state or assumptions. A new predictor can be accurate in isolation but incompatible with it. Test the behavior of both together and revert to the known pair if it fails. Predictor adaptation also requires insufficient reliability and enough data and budget. Automatic retraining each round or isolated component passes do not establish system improvement.
Illustrative example: If a predictor changes the action encoding order and the policy still expects the old order, both unit tests may pass while integration fails. Roll back the combination rather than just recording two passes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Implementation details
When may the predictor adapt, and why not update it after every failure?
RSI-Anything: Unlocking Recursive Self-Improvement with World Models
Show answer and example
Answer: Adaptation is triggered only when world-model reliability is insufficient, adaptation data are sufficient, and the remaining update budget covers the estimated cost. Failures do not necessarily come from the predictor, and updating on too few samples can chase noise. External verification does not remove the need for version attribution and pair compatibility checks. A short local run does not establish long-term self-improvement.
Illustrative example: A single erroneous prediction suggests unreliability, but without enough adaptation data or remaining budget, do not force training and call the resulting noisy update learning.
Sections in the Chinese source: 核心公式, 本地复现, 结果、验证与边界. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Method comparison
What is the trade-off between updating only the policy and adapting the world model too?
RSI-Anything: Unlocking Recursive Self-Improvement with World Models
Show answer and example
Answer: A fixed predictor gives a stable evaluator but may mislead persistently. Adaptation can address distribution shift, yet costs more and can alter interfaces the policy depends on. The framework gates adaptation by reliability, data, and budget and runs component and paired checks. Lower predictor loss does not by itself establish a better overall policy.
Illustrative example: The old workflow may read the wrong field once the predictor learns a new feedback format. Check the pair rather than just verifying that both components can run in isolation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Basics
Why does a world model's prediction still need external validation feedback?
RSI-Anything: Unlocking Recursive Self-Improvement with World Models
Show answer and example
Answer: A prediction can screen consequences in advance but can be wrong; external tools or the environment supply actual evidence. RSI-Anything uses feedback to update persistent policy or workflow and updates the predictor only if reliability is insufficient and adaptation data and budget are adequate. A high predicted score is not verified task success.
Illustrative example: A programmatic layout predicted to be valid still needs a trusted geometry checker. If the result conflicts, record the prediction error instead of using the prediction as verification.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Core mechanism
Why does execution feedback update only the LinUCB arm for the tool actually selected?
RATED: Reliability-Aware AI Agent Tool Retrieval via Execution Dynamics
Show answer and example
Answer: Only execution exposes success, error, or timeout. Unchosen outcomes are counterfactual. Per-tool updates avoid blaming all candidates for one failure, while discounting reduces old reliability evidence over time. Semantic top-K supplies candidates; exploration does not prove an uncalled tool is better.
Illustrative example: Example: If tool A times out, update A. Do not mark unused B successful; retain uncertainty until it is tried.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Implementation details
How does γ affect history for the selected tool in discounted LinUCB?
RATED: Reliability-Aware AI Agent Tool Retrieval via Execution Dynamics
Show answer and example
Answer: Decay the selected arm's diagonal A and b by γ, then add current squared features and execution feedback. Older evidence loses influence, and A also controls exploration uncertainty, so this is not just changing a success rate. The local encoder trained briefly, but the authorized tool binding failed; bandit updates are not tool success.
Illustrative example: Example: Recent timeouts can outweigh a long success history under decay. Permanent cumulative statistics may hide the change.
Sections in the Chinese source: 核心公式, 本地复现, 结果、验证与边界. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Method comparison
What do the frozen dual encoder and online discounted LinUCB adapt to?
RATED: Reliability-Aware AI Agent Tool Retrieval via Execution Dynamics
Show answer and example
Answer: The offline encoder supplies semantic query-tool representations and stays frozen in deployment. LinUCB updates each arm from actual feedback and discounts older evidence as reliability changes. Bandit reranking is not online retraining of the text encoder.
Illustrative example: Example: Successful calls after service recovery can raise arm value while encoder parameters remain unchanged.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Basics
Why rerank by execution reliability when a tool is semantically relevant?
RATED: Reliability-Aware AI Agent Tool Retrieval via Execution Dynamics
Show answer and example
Answer: The best matching description may refer to a tool that fails, times out, or is unavailable. RATED retrieves semantic Top-K first, then combines static metadata and per-tool online feedback. Reliability complements relevance without justifying a tool that cannot solve the task.
Illustrative example: Example: Favor the weather API that still works over a frequently timing-out equivalent, not an irrelevant reliable calculator.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why might subtracting the neighbor mean direction help find memories with unique facts?
NEAR: Neighborhood-Aware Residuals for Memory Retrieval
Show answer and example
Answer: Neighbors often share names and topics, making irrelevant entries similar. Projecting out the shared direction highlights distinguishing residuals. Queries use a precomputed reference table, not relevance labels to choose a direction. Residuals can remove useful shared clues too, so retain the original view and choose fusion on development data.
Illustrative example: Example: Two records mention the same business trip, but only one says the return is Friday. Suppressing the common trip direction can make the return fact matter more.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
Why not L2-normalize the whole concatenated vector after fusing layers?
NEAR: Neighborhood-Aware Residuals for Memory Retrieval
Show answer and example
Answer: Layer scores are scaled by memory–memory population variance and weighted. Concatenation coefficients make one dot product equal the fusion formula. Global normalization adds a sample-dependent denominator and changes ranking. Variance includes self-similarity, and neighborhoods are recomputed on normalized residuals at each layer rather than reused from layer one.
Illustrative example: Example: Two memories have different concatenated norms. Unit normalization penalizes the larger norm, so scores no longer equal the chosen linear combination.
Sections in the Chinese source: 核心公式, 本地复现, 代码映射与边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
How do removing a shared direction and training a query-only adapter differ in their information sources?
NEAR: Neighborhood-Aware Residuals for Memory Retrieval
Show answer and example
Answer: NEAR constructs memory references, scales, and its small table only from memory states, with no query labels. A query gets its direction from the anchor table. Supervised adapters use training relevance to update query transforms. These are different information contracts; an unlabeled reference did not learn an answer-specific direction.
Illustrative example: Example: Memory neighborhoods can be built without query answers. Choosing a reference because it is the correct evidence would introduce supervision leakage.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Why is the actual clue harder to find when many memories discuss the same person or project?
NEAR: Neighborhood-Aware Residuals for Memory Retrieval
Show answer and example
Answer: Whole-record representations can be dominated by common topics. Frequent names make records similar and bury rare dates or events. NEAR uses neighborhood relations to emphasize distinctions and retrieve specific facts rather than repeatedly retrieving topical material.
Illustrative example: Example: Ten records mention project A, but only one says it moved to Thursday. A date question needs that difference, not more mentions of the name.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Core mechanism
Why might faster individual decisions raise total resources per successful delivery?
When Faster Decisions Do Not Make Better Agents: An Empirical Study of Jev in Agent Workflows
Show answer and example
Answer: Errors can cause retries, failed recovery, or premature completion. Resource per success includes cost and time for every attempt divided by successes. Matched successes compare smooth paths only and cannot replace all-attempt delivery or hide failures.
Illustrative example: Example: A route half as expensive per attempt may succeed twice in ten tries, making it costlier per delivery than a slower route that succeeds eight times.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Implementation details
Why not compute a tidy ratio with zero successes, missing pairs, or invalid resource values?
When Faster Decisions Do Not Make Better Agents: An Empirical Study of Jev in Agent Workflows
Show answer and example
Answer: Cost per success is undefined at zero successes. Missing task/run pairs invalidate comparisons, and negative or nonfinite resources are invalid observations. Report or reject these states rather than delete failures to repair a denominator. The infrastructure consumes real paired traces and neither loads nor fabricates Jev weights or API outputs.
Illustrative example: Example: Do not add ε to an all-failure denominator or copy a paper row as a local run.
Sections in the Chinese source: 核心公式, 本地复现, 结果、验证与边界. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Method comparison
What do matched-success and all-attempt delivery each measure?
When Faster Decisions Do Not Make Better Agents: An Empirical Study of Jev in Agent Workflows
Show answer and example
Answer: Matched-success compares resources only on task/run pairs where both configurations succeed, isolating cost along successful paths. All-attempt includes failed resources and all successes to measure delivery. Conditioning on joint success can hide one configuration's many failures.
Illustrative example: Example: Three jointly solved tasks can show faster decisions without revealing failures among the seven others. Interpret both views.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Basics
Why evaluate cheap decisions per completed delivery instead of per invocation?
When Faster Decisions Do Not Make Better Agents: An Empirical Study of Jev in Agent Workflows
Show answer and example
Answer: Agents combine decisions, tools, and retries. Local savings can increase downstream failures and reruns. Jev Workflow counts all resources required for delivery so cheaper decisions and extra attempts share one denominator. Report different success subsets separately rather than advertise local latency alone.
Illustrative example: Example: Saving half a second per decision may cost three extra attempts. The user pays for all of them.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Core mechanism
Why compare experimental proposals in pairs and then fit value scores rather than choosing the most compelling description?
Janus: Enhancing Long-Horizon Autonomous Research with Co-Evolving World Models
Show answer and example
Answer: Pairwise comparisons establish a calibratable ordering for expected value, information gain, and exploration benefit. A sparse tournament fits Bradley–Terry scores to connect local judgments. Predictions allocate the experiment budget; executed successes and failures update the model and skills. A high predicted score is neither a completed experiment nor a substitute for execution.
Illustrative example: Example: Proposal A promises a larger metric gain, while B better rules out a key hypothesis. Adjust exploration weight to the remaining budget rather than treating polished promises as results.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Implementation details
How do soft comparisons in a sparse tournament become proposal value scores?
Janus: Enhancing Long-Horizon Autonomous Research with Co-Evolving World Models
Show answer and example
Answer: Fit the soft preferences on comparison edges by BCE on sigmoid(g_m-g_n), yielding relative Bradley–Terry scores, then weight the three benefits. This links local preferences without providing absolute success probabilities. The graph and skills update after execution; predictions cannot replace sandbox results.
Illustrative example: Example: If A slightly beats B and B beats C, fit their relative order. A's higher value does not mean its experiment has beaten the best current model.
Sections in the Chinese source: 核心公式, 本地复现, 结果、验证与边界. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Method comparison
How does selection by predicted gain alone differ from including information gain and exploration benefit?
Janus: Enhancing Long-Horizon Autonomous Research with Co-Evolving World Models
Show answer and example
Answer: Predicted gain alone favors currently promising regions and may overexploit them. Information gain values ruling out uncertain hypotheses, while exploration benefit gives unknown regions a chance. Janus adjusts weights with progress and remaining budget, but only actual execution can correct prediction errors.
Illustrative example: Example: A promises a small but well-understood gain. B may not improve the metric but can distinguish two mechanisms. Early in the budget, B may be worth testing first.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Basics
Why does Janus put unexecuted proposals in the research graph?
Janus: Enhancing Long-Horizon Autonomous Research with Co-Evolving World Models
Show answer and example
Answer: Unexecuted proposals are research directions that can compete for budget. The system estimates value, information gain, and exploration benefit before choosing what to run. The graph also stores actual experimental feedback and abstract skills so later proposals use prior evidence. Predictions guide selection; they do not count as completed experiments or observed scores.
Illustrative example: Example: A model change and a diagnosis of a prior failure can both enter as candidates. Remaining budget determines which one gets compute first.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Core mechanism
Without answers or rubrics, how does Propose–Probe–Compose avoid treating self-scores as a correctness oracle?
Harness²: Recursive Agent Harnessing for an Open World
Show answer and example
Answer: Candidates are actually executed before composing them using public tasks and behavioral differences. Evidence comes from traces and artifacts rather than hidden answers. Anonymize artifacts during selection to reduce preference based on harness identity. This open-ended improvement procedure does not guarantee correctness; selector, reflection, and probe calls must count toward cost, not just final executions.
Illustrative example: A general proposal may strengthen verification and a domain proposal may improve retrieval. Combine useful parts after running them rather than assuming that a self-declared winner completed the task correctly.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Implementation details
Why can the final selector only inspect anonymized artifacts, rather than harness identities or rewriting answers?
Harness²: Recursive Agent Harnessing for an Open World
Show answer and example
Answer: Anonymity reduces preference based on proposal origin. The selector can only choose from final artifacts actually executed, without reference answers, proposal intermediates, or newly generated answers. At k=1, take the sole result directly. Trusted local scripted probes verify the execution accounting, not open-ended LLM performance, and outer arithmetic checks cannot feed back into the editor or reflector.
Illustrative example: If the selector knows which output came from an 'advanced' plan, it might favor that identity. If it edits the answer, it is no longer choosing from completed artifacts.
Sections in the Chinese source: 核心公式, 本地复现与边界. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Method comparison
Why do parallel independent compositions and sequential recursive improvements have different candidate dependencies?
Harness²: Recursive Agent Harnessing for an Open World
Show answer and example
Answer: The parallel mode proposes general and domain candidates separately, then forms independent composites. Each sequential step reads the preceding harness and the within-task history, so it depends on earlier results. Both use 4k executions and 3k proposals; selectors and reflection are counted separately. Less waiting from parallelism does not reduce call cost.
Illustrative example: With k=2, parallel execution yields two independent final composites, whereas the second sequential attempt builds on the first. Candidate dependencies are therefore different.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Basics
Why does Harness² study improving agent workflows without reference answers or scoring rubrics?
Harness²: Recursive Agent Harnessing for an Open World
Show answer and example
Answer: Many real tasks specify a deliverable without providing a callable correctness oracle. Execution probes can still expose workflow weaknesses. Harness² studies constructing a better harness through proposal, probing, and composition. Observed changes or high self-ratings do not show that an answer is correct; final artifacts need selection and checks separate from generation.
Illustrative example: An open-ended research report has no unique answer. Probe source collection and organization and then inspect anonymized artifacts rather than trusting the author's self-assessment.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
Why does training the Proposer and Reflector with a frozen executor improve the evolver itself?
Evo2: Evolving the Evolver through Weight-Harness Co-Evolution
Show answer and example
Answer: The task model stays fixed, but parameters of the policies that propose and repair harnesses change. Effective edit trajectories provide training signals for both roles. Improvement targets how harnesses are changed, not just the candidates. Keep actions still earn zero; unchanged proposals are not successful innovations.
Illustrative example: The same executor handles tasks, but the trained proposer chooses better log compression and the reflector fixes bad hooks. The gain comes from changing the improvement process.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
How are keep rewards, role advantages, and truncated tokens handled in the loss?
Evo2: Evolving the Evolver through Weight-Harness Co-Evolution
Show answer and example
Answer: Keep earns zero. For each role, subtract the group mean reward. Normalize token loss by fixed Lmax and mask truncated outputs so length does not alter training scale. The real local path has specific environment and resource limits: a 7B policy/executor plus frozen copies is not presumed to fit A30 or constitute the full paper benchmark.
Illustrative example: Do not supervise an incomplete truncated tail as a correct edit. Averaging by the shorter actual length would also alter the original training weight.
Sections in the Chinese source: 核心公式, 本地复现, 结果、验证与边界. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
How does tree-parent weight ε+|V−1/2| differ from favoring only high-success nodes?
Evo2: Evolving the Evolver through Weight-Harness Co-Evolution
Show answer and example
Answer: It favors values far from the midpoint and retains a fixed root sampling share. This allocates search rather than monotonically choosing higher scores. Cross-branch crossover and the reflector also produce new harnesses; greedy champion-only search is not equivalent.
Illustrative example: V=0 and V=1 are both farther from the center than V=1/2, so parents are not selected simply in descending success-rate order.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why does Evo2 want proposal and reflection abilities to learn from search experience?
Evo2: Evolving the Evolver through Weight-Harness Co-Evolution
Show answer and example
Answer: A fixed evolver repeats the same proposal habits and may persist with low-value or invalid changes. Evo2 trains generation and reflection from search experience while freezing the task executor to clarify what updates. New search versions do not mean the base executor trained, and benefits still require independent execution evaluation.
Illustrative example: Train a proposer to learn from repeated invalid structures without necessarily changing the weights of the model that executes the tasks.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why is global token top-k followed by memory-ID aggregation different from per-memory MaxSim?
Beyond the Final Embedding: Retrieving Memories from Token-Level Evidence
Show answer and example
Answer: In global retrieval, tokens from all memories compete for a sparse return budget. MaxSim per record searches matches independently, changing computation and candidate distributions. Unreturned positions use a shared floor rather than true zero evidence. The pooled branch preserves overall meaning and can surface a memory without retrieved local tokens.
Illustrative example: Example: One record contains a rare date that strongly matches the question. Global token search can retrieve the date then its record without first ranking the pooled record highest.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
Why doesn't the query adapter backpropagate through discrete neighbor selection?
Beyond the Final Embedding: Retrieving Memories from Token-Level Evidence
Show answer and example
Answer: Training uses multi-positive cross-entropy on all-memory MaxSim as a continuous surrogate; inference uses sparse token search. Discrete ANN selection does not supply a valid gradient. The low-rank residual adjusts only query content tokens while preserving norm. Encoder, memory bank, and pooled query stay frozen; validation alone chooses hyperparameters.
Illustrative example: Example: Top-K indices jump when a query changes, so they cannot be differentiated as a smooth function. Train on differentiable full matching, then use the sparse index at inference.
Sections in the Chinese source: 核心公式, 本地复现, 数据协议、结果与边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
Which representations change with a query-only adapter versus retraining the memory encoder?
Beyond the Final Embedding: Retrieving Memories from Token-Level Evidence
Show answer and example
Answer: Query-only training freezes the encoder and memory bank and applies a low-rank residual to query content tokens, adapting them to existing geometry. Retraining the encoder can move both sides and require re-encoding old memory under a consistent version. The smaller update cannot restore information already lost in encoding.
Illustrative example: Example: Keep a million memory vectors fixed and adjust the query for who approved on Tuesday. Updating the shared encoder may require recomputing all old vectors.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Why can a pooled embedding miss a local fact in memory?
Beyond the Final Embedding: Retrieving Memories from Token-Level Evidence
Show answer and example
Answer: Pooling compresses several facts into one vector, so a date or object difference can be drowned out by a shared topic. Final-layer content tokens retain finer contextual evidence for sparse matching. A token match is not an independent dictionary meaning or guaranteed fact; combine it with the global branch.
Illustrative example: Example: Two records discuss travel, but only one sentence states a rebooking date. Their pooled vectors can be close while the local date token distinguishes them.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
Why use zero-sum KL coefficients instead of adding KL loss?
When KL Regularization Misfires in Group Policy Optimization
Show answer and example
Answer: Independent KL can update when reward gradients cancel or rewards tie. Center average conditional-KL drift with group rewards and feed the existing surrogate. Skip zero-scale/ineligible groups, including optimizer side effects, rather than returning zero loss and stepping.
Illustrative example: All rewards equal gives no relative signal; verify no optimizer update rather than rely on the logged loss.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
Why skip optimizer updates for all-equal rewards in ZCPO?
When KL Regularization Misfires in Group Policy Optimization
Show answer and example
Answer: Reward scale is zero, and AdamW decay may move parameters even with a zero loss. Skip the group/step; undersized groups also cannot form relative coefficients.
Illustrative example: Zero logged gradient plus weight decay can still alter the model.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
Can a sampled-token log ratio substitute for ZCPO conditional KL?
When KL Regularization Misfires in Group Policy Optimization
Show answer and example
Answer: No. Use full-vocabulary conditional KL, averaged along each response and centered across group. A sampled ratio is a different noisy estimator.
Illustrative example: A rare sampled token's drift does not describe the entire conditional distribution.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
Why put KL drift into ZCPO group coefficients rather than an independent loss?
When KL Regularization Misfires in Group Policy Optimization
Show answer and example
Answer: A standalone KL can update policy with no reward distinction. Center relative KL and rewards into zero-sum coefficients for the existing surrogate and relative group credit.
Illustrative example: Equal rewards should not trigger a training step merely because a separate KL exists.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Core mechanism
Why restore round-start weights and retrain only selected skills?
Which Skill to Distill? SGUID: Selecting a Compact Skill Bank for Model–Skill Co-Evolution
Show answer and example
Answer: Otherwise broad probe exposure confounds the compact curriculum effect. Reset, train selected skills, and regenerate from the updated model next round. Count sustained verifier-aligned teaching, not semantic matches.
Illustrative example: Testing ten hints gives extra practice; without a reset, later gain can't be assigned to the two selected hints.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Implementation details
May SGUID delete negative coefficients, and how should negative results be handled?
Which Skill to Distill? SGUID: Selecting a Compact Skill Bank for Model–Skill Co-Evolution
Show answer and example
Answer: No. The objective allows negative teaching signals. Check early/late activity and quality. Validation decline after two executed rounds remains a negative result, not success because the loop ran.
Illustrative example: Early gains don't justify retention if late signals fall below thresholds.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Method comparison
Why restore initial round weights after SGUID probing?
Which Skill to Distill? SGUID: Selecting a Compact Skill Bank for Model–Skill Co-Evolution
Show answer and example
Answer: Probes already update the model. Restore to isolate training with compact selected library from extra probe learning. Next round generates new skills, not a fixed manual archive.
Illustrative example: Continuing after probing all candidates would retain discarded skills' effects.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Basics
Why measure sustained teaching activity rather than similarity in SGUID?
Which Skill to Distill? SGUID: Selecting a Compact Skill Bank for Model–Skill Co-Evolution
Show answer and example
Answer: Relevant skills can teach poorly. Generate from verified traces and check early/late persistence and verifier agreement. Retrieval frequency isn't useful teaching count.
Illustrative example: A frequently retrieved but misleading skill shouldn't stay because it looks relevant.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Core mechanism
Why recompute student probabilities and advantages despite cached teacher scores?
When Do We Need On-Policy Distillation? Distilling on Offline Student Rollouts Is Often Better
Show answer and example
Answer: Fixed trajectories and teacher outputs can be reused, but student probabilities and gaps change after updates. Stale directions distort the objective. Offline student rollouts save compute, not fresh on-policy sampling every step.
Illustrative example: A once-underweighted token may exceed teacher support; old positive credit would still push it up.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Implementation details
What must remain immutable in the Semi-OPD cache?
When Do We Need On-Policy Distillation? Distilling on Offline Student Rollouts Is Often Better
Show answer and example
Answer: Align prompts, responses, teacher scores and masks. Refresh only current student scores and detached advantages each step. This doesn't guarantee long-term coverage.
Illustrative example: Deleting a token while reusing old indexed scores misaligns supervision.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Method comparison
Why cache teacher labels but not advantages?
When Do We Need On-Policy Distillation? Distilling on Offline Student Rollouts Is Often Better
Show answer and example
Answer: Frozen teacher scores remain valid for fixed prefixes. Advantage also depends on current student logp and must refresh. A cached advantage is a stale objective.
Illustrative example: When student support rises, the teacher gap shrinks despite unchanged teacher score.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Basics
Why fix trajectories generated by the initial student in Semi-OPD?
When Do We Need On-Policy Distillation? Distilling on Offline Student Rollouts Is Often Better
Show answer and example
Answer: One collection and teacher-scoring pass saves future costs. Train on fixed visited states, while acknowledging later policy drift means data are no longer freshly on-policy.
Illustrative example: Reuse initially common error prefixes without collecting newer ones.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Core mechanism
Why subtract expected residual under the student and then center per response?
Residual Advantage: Student-Relative Teacher Guidance for RL with Verifiable Rewards
Show answer and example
Answer: Raw differences depend on student coverage. Student expectation yields relative local credit; response centering separates positional effects. Add to verifier advantage, not replace it. Fix scoring across the batch and refresh teacher only next round.
Illustrative example: One word's positive residual isn't sufficient if alternatives have similarly positive residuals.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Implementation details
Why can't Co-RA change teachers midway through a scored batch?
Residual Advantage: Student-Relative Teacher Guidance for RL with Verifiable Rewards
Show answer and example
Answer: Student/teacher consume the same scored batch. Teacher LoRA uses fixed verifier advantages, and the new teacher is used next round. Relabeling midway changes targets and provenance.
Illustrative example: Old teacher on the first half and updated teacher on the second is not fixed supervision.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Method comparison
How do residual advantages differ from full-distribution KL distillation?
Residual Advantage: Student-Relative Teacher Guidance for RL with Verifiable Rewards
Show answer and example
Answer: Bound and center local advantages to feed policy updates, while KL directly constrains distribution differences. Gradients and reward combination differ despite using teacher probabilities.
Illustrative example: Subtract student expectation and response mean rather than copying logits.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Basics
Why turn teacher–student probability residuals into local advantages?
Residual Advantage: Student-Relative Teacher Guidance for RL with Verifiable Rewards
Show answer and example
Answer: Teacher differences supply dense guidance. Subtract the student-distribution expectation to get relative local credit, then combine with verifier advantages rather than only terminal teacher scores.
Illustrative example: A teacher preferring another word at the same prefix suggests a local update.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Core mechanism
Why measure pruning damage on a probe but reselect channels on the unpruned model?
ReCal: Calibrating Structured Pruning for On-Policy Distillation Recovery
Show answer and example
Answer: Probe reveals harmed teacher states; use their damage to reweight original activations. This can avoid poor student trajectories that are hard to repair. Preserve remaining vocabulary mass in damage KL.
Illustrative example: Raise calibration weight on carry states harmed by the probe instead of patching already removed channels arbitrarily.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Implementation details
May ReCal drop probability outside teacher top candidates and renormalize?
ReCal: Calibrating Structured Pruning for On-Policy Distillation Recovery
Show answer and example
Answer: No. Keep top events and remainder mass to estimate damage. Normalize response damage weights to unit scale; zero damage becomes uniform. Recompute selection from unpruned stats.
Illustrative example: Movement outside top tokens is invisible after local renormalization.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Method comparison
How does damage-weighted calibration differ from activation-size pruning?
ReCal: Calibrating Structured Pruning for On-Policy Distillation Recovery
Show answer and example
Answer: Probe teacher states for damage, then reweight unpruned model statistics. Activation magnitude alone ignores recovery. Match target width and restoration protocol for fair comparison.
Illustrative example: A rare but badly damaged state can gain calibration weight.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Basics
Why account for recoverability before pruning in ReCal?
ReCal: Calibrating Structured Pruning for On-Policy Distillation Recovery
Show answer and example
Answer: Severe pruning can induce poor student states that distillation struggles to repair. Probe damage informs channel reselection toward recoverable structures instead of relying solely on later training.
Illustrative example: Removing a key arithmetic path may not be fixed by simply distilling longer.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Core mechanism
Why route low-perplexity problems to RL earlier?
PIVOT: Perplexity-Informed KD-to-RL Transition Scheduling for Vertical-Domain Few-Shot Distillation
Show answer and example
Answer: Teacher support on actual student responses indicates readiness for reward exploration. High perplexity retains supervision; aggregate by prompt and ramp RL share. This isn't a correctness or intrinsic difficulty test.
Illustrative example: Switch familiar teacher-supported steps sooner, while frequently divergent prompts continue distillation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Implementation details
How turn PIVOT response scores into problem routing?
PIVOT: Perplexity-Informed KD-to-RL Transition Scheduling for Vertical-Domain Few-Shot Distillation
Show answer and example
Answer: Exponentiate each response's mean teacher negative logp, average per prompt, standardize within batch and choose lowest via cosine proportion. Preserve separate GRPO/OPD normalizations, not token-level routing.
Illustrative example: Don't decide a whole question from only its easiest completion.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Method comparison
Why aren't teacher perplexity and verified correctness interchangeable?
PIVOT: Perplexity-Informed KD-to-RL Transition Scheduling for Vertical-Domain Few-Shot Distillation
Show answer and example
Answer: Perplexity measures teacher support for student wording, not task result. High support can be wrong and low support an effective novel route. It is a routing proxy.
Illustrative example: Teacher unfamiliarity with phrasing does not make the conclusion false.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Basics
Why switch PIVOT prompts to RL individually?
PIVOT: Perplexity-Informed KD-to-RL Transition Scheduling for Vertical-Domain Few-Shot Distillation
Show answer and example
Answer: Students are not equally ready on all prompts. Uniform switching removes guidance too early for hard cases. Rank teacher perplexity on student replies and expand RL among low-perplexity prompts.
Illustrative example: Let familiar tasks explore rewards while unfamiliar ones still distill.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Core mechanism
Why doesn't high residual OPD loss guarantee learnability?
Why On-Policy Distillation Sometimes Fails: Vanishing Learning Signals
Show answer and example
Answer: A numeric gap differs from gradient that can move parameters; saturation/coverage may weaken it. Jointly inspect pre-clip gradients, split estimates, parameter movement and representations. Nonpositive denominators are undefined. Diagnostics aren't causes.
Illustrative example: Loss can remain high while gradients and parameter updates are nearly zero.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Implementation details
What if a learning-signal proxy has nonpositive loss or split denominator?
Why On-Policy Distillation Sometimes Fails: Vanishing Learning Signals
Show answer and example
Answer: Mark the ratio undefined, not zero-filled. Compute CKA from per-prompt averages centered across prompts at specified precision. Use gradients before clipping.
Illustrative example: A negative split gradient dot product should not become a positive occupancy by absolute value.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Method comparison
Do occupancy and CKA diagnostics prove why training stalled?
Why On-Policy Distillation Sometimes Fails: Vanishing Learning Signals
Show answer and example
Answer: No. They describe patterns among sampling, updates and representations. Causal attribution needs controls or assumptions.
Illustrative example: Similar layers and slow learning occurring together don't establish one caused the other.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Basics
Why inspect both loss and gradients in OPD Learning Signals?
Why On-Policy Distillation Sometimes Fails: Vanishing Learning Signals
Show answer and example
Answer: Large residual divergence can coexist with weak updates. Combine loss, pre-clip gradients, parameter motion and representation similarity to diagnose stagnation.
Illustrative example: Distributions can differ while current parameterization yields little usable gradient.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Core mechanism
Why need a differentiable virtual update instead of weighting by current loss?
MetaOPD: Meta-Learned Token Weighting for On-Policy Distillation
Show answer and example
Answer: High loss can be noise. A virtual step reveals downstream reference-loss effects and meta-gradients, then real student update uses learned weights. Isolate descriptors, optimizer state and training-only references.
Illustrative example: Compare whether emphasizing a stylistic high-loss word or a reasoning token better serves the reference task.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Implementation details
What detaches in MetaOPD's 74-dimensional descriptors and weight transform?
MetaOPD: Meta-Learned Token Weighting for On-Policy Distillation
Show answer and example
Answer: Detach descriptors, not the weight network through the virtual step. Center per response, bound via tanh and normalize to mean one. Use new weights for real update; references come from allowed training data.
Illustrative example: Test references cannot supervise outer-loop weights.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Method comparison
How does differentiable virtual AdamW differ from scoring an actual step afterward?
MetaOPD: Meta-Learned Token Weighting for On-Policy Distillation
Show answer and example
Answer: Virtual step retains the graph to weight network for meta-gradients, while real update mutates state without that derivative. Optimizer state/order matter; arbitrary SGD isn't equivalent.
Illustrative example: Ask how weights caused the hypothetical change, not only whether a metric changed.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Basics
Why learn MetaOPD weights through a virtual student step?
MetaOPD: Meta-Learned Token Weighting for On-Policy Distillation
Show answer and example
Answer: Teacher gap does not reveal which signals improve tasks. Evaluate virtual updates against training-reference loss, use meta-gradients to learn weighting, then update the student with new weights.
Illustrative example: A large gap on a token may harm the reference task and deserve lower weight.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Core mechanism
Why can't subproblem effectivity replace final Goldilocks selection?
MagicBeanStalk: Escaping the RLVR Zero Advantage Trap with Recursive Data Synthesis
Show answer and example
Answer: Parent improvement checks decomposition on one learner. Final task must not be all-correct or all-wrong under target budget and needs normally completed correct trace. Local teacher probes don't complete recursive construction.
Illustrative example: Helpful subproblems for one student do not make a composed task learnable if target never solves it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Implementation details
Can a well-formed local MBS problem be considered accepted?
MagicBeanStalk: Escaping the RLVR Zero Advantage Trap with Recursive Data Synthesis
Show answer and example
Answer: No. Need complete correct solution, independent validation, blueprint/novelty and effectivity/Goldilocks checks. Retain real construction failures; isolated teacher diagnostics aren't acceptance.
Illustrative example: Valid JSON and dependencies do not rescue an unsolvable task.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Method comparison
What do subproblem effectivity and target-policy Goldilocks each verify?
MagicBeanStalk: Escaping the RLVR Zero Advantage Trap with Recursive Data Synthesis
Show answer and example
Answer: Effectivity asks if subexamples improve a student's parent success. Goldilocks requires mixed outcomes and at least one normally terminated success under target budget. One doesn't replace the other.
Illustrative example: An auxiliary learner may improve while target still fails every final sample.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Basics
Why recursively decompose and compose MagicBeanStalk problems?
MagicBeanStalk: Escaping the RLVR Zero Advantage Trap with Recursive Data Synthesis
Show answer and example
Answer: All-fail hard tasks provide no advantage. Effective subproblem examples reveal dependency blueprints to synthesize new leaves and compose tasks with mixed current-policy outcomes. Valid format isn't efficacy.
Illustrative example: Practice manageable parts and recombine them without losing key dependencies.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Core mechanism
Why filter high-probability positives but retain all nonpositives?
GRPODropout: Less is More for Online Reinforcement Learning Rollouts
Show answer and example
Answer: Dominant successes can further reduce entropy; select by entropy contribution, not random removal. Preserve negative learning and recenter with old probabilities. Generation cost is already paid.
Illustrative example: Downweight repeated successful templates without turning failures into omitted feedback.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Implementation details
Can GRPODropout keep original centered advantages after filtering?
GRPODropout: Less is More for Online Reinforcement Learning Rollouts
Show answer and example
Answer: No. Restore all nonpositive samples and recenter retained ones by old-policy probability. Dropping either condition changes feedback/baseline.
Illustrative example: Removing strong positives shifts the reference and requires recentering.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Method comparison
How does response dropout differ in cost from generating fewer replies?
GRPODropout: Less is More for Online Reinforcement Learning Rollouts
Show answer and example
Answer: Generate full group then filter updates; fewer samples would reduce generation and coverage, changing estimator. Less backward work isn't equal end-to-end savings.
Illustrative example: Using six of eight generated replies does not cost only six generations.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Basics
Why drop some high-probability positive-advantage replies in GRPODropout?
GRPODropout: Less is More for Online Reinforcement Learning Rollouts
Show answer and example
Answer: They can oversharpen common modes. Selectively drop by entropy contribution while retaining failures. It isn't neuron dropout and doesn't refund generated rollouts.
Illustrative example: Reinforce fewer repeats of a common successful template.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Core mechanism
Why not select solely by largest teacher–student logp gaps?
DIAL-OPD: Learning More from Fewer Tokens in On-Policy Distillation
Show answer and example
Answer: Huge gaps can occur in negligible mass. Logarithmic probability mean rescales priorities. Keep masked tokens as context; sparse supervision does not reduce generation or prove errors.
Illustrative example: An unlikely odd word should not drown out disagreement on a realistic key step.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Implementation details
How handle equal p and q in DIAL's logarithmic mean?
DIAL-OPD: Learning More from Fewer Tokens in On-Policy Distillation
Show answer and example
Answer: Use its continuous limit. Choose each response's own valid-position quota, detach advantages and differentiate final student logp only. Global ranking can let long answers dominate.
Illustrative example: Equal probabilities have a finite limit, not NaN or an arbitrary large score.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Method comparison
How does sparse DIAL supervision differ from deleting unselected text?
DIAL-OPD: Learning More from Fewer Tokens in On-Policy Distillation
Show answer and example
Answer: Unselected tokens stay as context but have masked losses. Removing text changes later prefixes and teacher scores.
Illustrative example: Keep preceding arithmetic/connectives even when not training their tokens.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Basics
Why multiply DIAL-OPD log gaps by logarithmic probability mean?
DIAL-OPD: Learning More from Fewer Tokens in On-Policy Distillation
Show answer and example
Answer: Tokens near impossible under both models can have large log differences. Scaling downweights such noise and prioritizes supported disagreements. Selection is not verifier truth.
Illustrative example: A bizarre word differing tenfold may matter less than a small gap on a key digit.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Core mechanism
Why split WOVEN by initial scene or source rather than random QA rows?
WOVEN: Weaving Visual World Modeling into Multimodal LLMs
Show answer and example
Answer: One image can yield several actions, questions, and counterfactuals. Random row splits can put nearly identical states in both sets. Grouped isolation reduces shortcuts, and scene, action, and reasoning transfer need separate checks.
Illustrative example: Group two questions about different pushes on the same table rather than splitting their common source across train and test.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Implementation details
What do four option permutations and source_id grouping guard against?
WOVEN: Weaving Visual World Modeling into Multimodal LLMs
Show answer and example
Answer: Move the correct letter with its option and image to prevent letter shortcuts. Split by initial source to prevent related variants leaking across sets. The prompt exposes only question, options, and actual images, not answer-related distractor_type. Local training runs SFT, not GRPO.
Illustrative example: Put three questions from one initial frame together in train or holdout so source memorization cannot masquerade as test generalization.
Sections in the Chinese source: 核心公式, 本地复现, 代码映射与边界. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Method comparison
What distinct generalization problems do scene-OOD and state perturbation test?
WOVEN: Weaving Visual World Modeling into Multimodal LLMs
Show answer and example
Answer: Scene-OOD changes scene distribution; state perturbation changes state and tests robustness. High in-distribution accuracy does not solve either. The paper also controls action and reasoning axes, and per-benchmark subsets do not establish one universal model gain.
Illustrative example: A new restaurant can be handled well while tiny object-position shifts in one scene still cause errors.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Basics
Why isn't WOVEN transition supervision ordinary static classification?
WOVEN: Weaving Visual World Modeling into Multimodal LLMs
Show answer and example
Answer: It models relations among initial state, action, and outcome, including forward prediction, inverse action, and counterfactuals. Generated video rollouts supply visual transitions and intermediate frames, not outcomes stated in prompts. Recognizing object classes is insufficient.
Illustrative example: The same ball starts in one place but moves differently for left and right pushes; choose the action or outcome, not whether there is a ball.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Core mechanism
Why does TRACE UPDATE replace the active Fine set rather than accumulate it?
Look Back, Think Ahead: Visual Memory on Demand for Efficient Multimodal Reasoning
Show answer and example
Answer: Accumulation would let active context grow beyond budget. ViMoD retains Coarse while current generation history decides whether to keep or replace Fine. This changes the attention working set, not permanent storage of recoverable Fine details.
Illustrative example: Replace the left-label working set with a right gauge while keeping the overview and ability to retrieve the label later.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Implementation details
How do gradients cross DART groups when forward assignments are hard?
Look Back, Think Ahead: Visual Memory on Demand for Efficient Multimodal Reasoning
Show answer and example
Answer: Constrained capacities backpropagate through an implicit Jacobian. A hard+soft−stopgrad(soft) estimator uses hard assignments forward and Sinkhorn continuous capacities backward. Stage training needs real Fine-index labels or an online teacher and must fail if missing.
Illustrative example: A patch is assigned to group A in forward, but continuous assignments can adjust capacity backward. A hardcoded answer region is not learned retrieval evidence.
Sections in the Chinese source: 核心公式, 本地复现, 代码映射与边界. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Method comparison
How do DART's learned capacities differ from fixed-size pooling?
Look Back, Think Ahead: Visual Memory on Demand for Efficient Multimodal Reasoning
Show answer and example
Answer: Fixed pooling assigns equal position counts to all regions. DART learns region sizes and memberships under a total-capacity constraint, using continuous capacities during training. It adapts granularity but remains budgeted and cannot guarantee every detail survives.
Illustrative example: Group a large blank area together and keep more groups for dense text; a fixed grid cannot adapt that way.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Basics
Why recover original detail after ViMoD compresses the image?
Look Back, Think Ahead: Visual Memory on Demand for Efficient Multimodal Reasoning
Show answer and example
Answer: A need for small text or local evidence may emerge mid-generation. ViMoD keeps a compact overview and a recoverable detail cache, and text state selects what to read. This is neither permanent lossy compression nor reading every detail on every step; retrieval has its own costs.
Illustrative example: Recognize a receipt first, then recover the tax field when needed.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Core mechanism
Why can't visual compression for Gated DeltaNet reuse softmax attention weights directly?
V-CoLA: Vision Token Compression with Linear Attention
Show answer and example
Answer: Linear recurrence does not expose the same full attention matrix. State updates and forgetting determine what survives. V-CoLA measures final-state readback for long-term retention and short-window response change for uniqueness, then merges contiguous blocks. These are not ordinary similarity scores.
Illustrative example: An initially salient patch may be overwritten and score poorly on final readback even if local similarity was high.
Sections in the Chinese source: 背景与主要改动, 核心公式与实现选择. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Implementation details
Why document the transposed state and query probe choices?
V-CoLA: Vision Token Compression with Linear Attention
Show answer and example
Answer: The paper uses value×key while the official backend and local code use key×value, requiring transposed recurrence operations. Equation 7 uses a query probe and the appendix shifted keys; local chooses the former and gates on actual state replay with the official kernel. Equivalence is not assumed.
Illustrative example: Reversed axes can yield legal shapes but wrong reads, so numerical replay catches errors shape checks miss.
Sections in the Chinese source: 核心公式与实现选择, 本地复现, 复现边界与 Evolve. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Method comparison
What different omissions do long-term retention and short-term uniqueness address?
V-CoLA: Vision Token Compression with Linear Attention
Show answer and example
Answer: Long-term readback asks whether a token's value remains in final state. Short-term response change asks whether its update adds distinct information. Combine them 0.9/0.1 before contiguous block merging rather than deleting by static similarity alone.
Illustrative example: One token may remain useful long term while another contributes a short-lived detail; neither signal alone captures both.
Sections in the Chinese source: 背景与主要改动, 核心公式与实现选择. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Basics
Why need visual compression for hybrid linear-attention VLMs?
V-CoLA: Vision Token Compression with Linear Attention
Show answer and example
Answer: Many vision tokens consume language-layer compute and state, but linear attention stores evidence differently from softmax. V-CoLA uses actual recurrence to select what to merge or skip at deeper layers. Token count alone does not establish answer quality or inference savings.
Illustrative example: Compress redundant high-resolution background patches while preserving a small sign the question needs.
Sections in the Chinese source: 背景与主要改动, 核心公式与实现选择. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Core mechanism
Why does UMRM compare all B² positive–negative pairs?
Unsupervised Multimodal Reward Modeling
Show answer and example
Answer: All positives and negatives in the batch contribute ranking constraints instead of isolated pairs. Pool only response tokens so prompt/image features do not swamp response distinctions. Synthetic negatives define training signals but do not cover every real policy hallucination.
Illustrative example: Two positives and two negatives give four comparisons rather than two matched pairs, without creating human preference labels.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Implementation details
Should UMRM change the image to match a perturbed negative description?
Unsupervised Multimodal Reward Modeling
Show answer and example
Answer: No. The external sampler edits the response while holding the image fixed. It can change a span, splice tails, shuffle clauses, or use a cross-image caption with default probability 0.15. Swapping the image changes the contrast. Do not pool image or prompt tokens into the response representation.
Illustrative example: Change “two dogs” to “three dogs,” but keep the original image with two dogs.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Method comparison
Why can caption-perturbation accuracy and real-policy selection utility move in opposite directions?
Unsupervised Multimodal Reward Modeling
Show answer and example
Answer: Synthetic negatives may differ from actual policy errors. A reward head can learn artifacts of artificial editing. Bootstrapping on policy candidates with a verifier changes the supervision and may improve Best-of-N even if a reward benchmark falls.
Illustrative example: Spotting swapped colors does not ensure that a reward head can select a faithful response when policy candidates omit objects.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Basics
Does UMRM's unsupervised label mean no labels are used anywhere?
Unsupervised Multimodal Reward Modeling
Show answer and example
Answer: No. Caption perturbations construct preference pairs without human pairwise annotation. Layer selection uses a labeled development set, and bootstrapping uses cheap verifier labels. Preference construction, model selection, and deployment-utility supervision differ.
Illustrative example: Change “two cups” to “three cups” automatically, but validation labels can still determine the best layer.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Core mechanism
Why does target stop-gradient not freeze the entire vision tokenizer?
Mid-Training Language Models on Raw Video
Show answer and example
Answer: Detaching targets prevents them from moving or collapsing with predictions. The input-side tokenizer still receives gradients through the LM and prediction head. Caption-free midtraining learns visual sequence relations and needs matched instruction tuning for comparison.
Illustrative example: The next patch is detached as target, but prediction error still updates the tokenizer through earlier input patches.
Sections in the Chinese source: 背景与主要改动, 核心公式与原文效果. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Implementation details
Why match image-text instruction tuning after video midtraining?
Mid-Training Language Models on Raw Video
Show answer and example
Answer: Final ability depends on both stages. Different later tuning would confound the raw-video effect. The paper matches later tuning, so text-retention conclusions apply to that protocol, not arbitrary captionless training. A local component is not the full comparison.
Illustrative example: Giving only the video arm extra high-quality instructions would confound a later visual QA gain.
Sections in the Chinese source: 核心公式与原文效果, 本地复现与成本决策. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Method comparison
What supervision bottleneck does raw video avoid compared with captions?
Mid-Training Language Models on Raw Video
Show answer and example
Answer: Captions capture only described information, often omitting continuous motion and detail. Continuous token prediction uses raw order but costs longer sequences and vision regression. Better semantics are not guaranteed; match subsequent instruction tuning.
Illustrative example: “A ball moves” omits path and speed variation; raw video contains them, but downstream tests show whether the model uses them.
Sections in the Chinese source: 背景与主要改动, 核心公式与原文效果. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Basics
Why can uncaptioned video provide a useful language-model midtraining signal?
Mid-Training Language Models on Raw Video
Show answer and example
Answer: Ordered frames contain objects, motion, and spatiotemporal relationships not fully captured by text. The method turns frame patches into continuous sequences and predicts future representations causally. This is not captioning or proof of physical understanding; test after matched tuning.
Illustrative example: Video of lifting a cup preserves motion that the caption “someone picks up a cup” may omit.
Sections in the Chinese source: 背景与主要改动, 核心公式与原文效果. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Core mechanism
Why align adjacent Value coordinates before sharing, and why use full V for the current token?
VFold: Symmetry-Aware Cross-Layer Value Cache Compression
Show answer and example
Answer: CCA and head matching align spaces and fold transforms into V/output weights. Compute full attention for the current token before cross-layer sharing, while keys and protected ends remain independent. Invertible alignment does not make subsequent averaging lossless.
Illustrative example: Direct averaging can cancel equivalent features in different coordinates; aligned averaging is still approximate.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Implementation details
Why fold inverse transforms into every relevant GQA query output slice?
VFold: Symmetry-Aware Cross-Layer Value Cache Compression
Show answer and example
Answer: Several query heads share each physical KV head. Apply inverse transforms across all matching output slices. Current token uses full V first, and protected ends add overhead for short sequences.
Illustrative example: Correcting only one of two query groups leaves the other in mismatched coordinates.
Sections in the Chinese source: 核心公式, 数据协议与结果. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Method comparison
How would sharing keys as well as values change VFold?
VFold: Symmetry-Aware Cross-Layer Value Cache Compression
Show answer and example
Answer: It would change position matching, not just retrieved content. The documented method keeps keys separate and shares aligned values only.
Illustrative example: Layer weights over positions can differ even when some values are shared.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Basics
What is shared between key and value caches in VFold?
VFold: Symmetry-Aware Cross-Layer Value Cache Compression
Show answer and example
Answer: Keys match historical positions; values supply content. Only aligned adjacent-layer values are merged; keys and protected first/last tokens remain separate.
Illustrative example: Two layers can share processed content while retaining independent keys.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Core mechanism
Why calibrate SparseDecoding on actual decode activations and exclude prefill?
SparseDecoding: Decoding-Aware Pruning for Accurate and Efficient LLM Inference
Show answer and example
Answer: Deployment generates its own prefixes, whose states differ from teacher-forced and prefill states. Collect genuine dense autoregressive activations before pruning. Collection protocol is separate from SparseGPT or Wanda backend choices.
Illustrative example: A self-generated departure from the reference can use channels that teacher-forcing calibration never sees.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Implementation details
Why verify bit counts and value order in 32-column sparse packs?
SparseDecoding: Decoding-Aware Pruning for Accurate and Efficient LLM Inference
Show answer and example
Answer: At 50% sparsity each group must contain 16 set bits and values packed in increasing retained-column order. Mismatches read wrong weights. Dense zero-multiplication is not sparse SpMV.
Illustrative example: Packed value two maps to the second set bit, not original column two.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Method comparison
Does local Wanda use the same optimization as SparseGPT?
SparseDecoding: Decoding-Aware Pruning for Accurate and Efficient LLM Inference
Show answer and example
Answer: No. Wanda ranks N:M entries using weight magnitude times activation statistics, without Hessian inversion or error compensation. Sparsity does not make them equivalent.
Illustrative example: Both can keep half the weights but retain different entries and handle error differently.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Basics
Why separate activation collection from the pruning algorithm?
SparseDecoding: Decoding-Aware Pruning for Accurate and Efficient LLM Inference
Show answer and example
Answer: The protocol collects actual autoregressive decode activations and passes them to a backend. Substituting Wanda does not implement SparseGPT's optimization.
Illustrative example: Different pruners can consume the same decode activations.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Structured Decisions / Typed Readout: From Generated Sentences to Candidate Distributions
Core mechanism
Why do cached candidates and options in the request not imply full candidate interaction?
Show answer and example
Answer: Shared dual towers independently encode request and candidate and match normalized last-token vectors. A candidate can be cached because it doesn't depend on the request. Training partly drops closed-set options to discourage surface matching. Cross-reasoning remains limited, and cache requires identical model, prompt and candidate content.
Illustrative example: Example: Pre-encode tools and encode each request online. A changed tool definition invalidates its old vector.
Sections in the Chinese source: 背景与主要改动, 核心公式与原文效果. The example illustrates the mechanism.
Structured Decisions / Typed Readout: From Generated Sentences to Candidate Distributions
Implementation details
Why deduplicate open-retrieval negatives over the full MetaEncoder batch?
Show answer and example
Answer: Open retrieval uses global batch candidates. Treating duplicates as independent negatives changes training mass. The specified direction is request-to-candidate; adding a reverse term changes the objective.
Illustrative example: Example: The same tool appearing twice should not become two competing negatives.
Sections in the Chinese source: 背景与主要改动, 核心公式与原文效果. The example illustrates the mechanism.
Structured Decisions / Typed Readout: From Generated Sentences to Candidate Distributions
Method comparison
What does MetaEncoder's dual encoder sacrifice relative to joint deep interaction?
Show answer and example
Answer: Separate encoding enables caching and large retrieval, but limits fine-grained pairwise reasoning. Including options in a closed-set request adds context without turning the candidate tower into an arbitrary cross-encoder.
Illustrative example: Example: Detecting a subtle contradiction between long proofs can be harder with fixed vectors than joint reading.
Sections in the Chinese source: 背景与主要改动, 核心公式与原文效果. The example illustrates the mechanism.
Structured Decisions / Typed Readout: From Generated Sentences to Candidate Distributions
Basics
How does MetaEncoder use one representation for closed choices and open retrieval?
Show answer and example
Answer: Encode requests and candidates separately with a shared encoder, normalize their final-token embeddings and compare similarity. Closed choice uses supplied options; open retrieval searches a larger bank. Candidate scope and training negatives differ.
Illustrative example: Example: One request vector can choose among three tools or search a full description index.
Sections in the Chinese source: 背景与主要改动, 核心公式与原文效果. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Core mechanism
Why prevent LeWAM's noisy-action branch from seeing clean actions indirectly?
LeWAM: End-to-End World and Action Modeling with JEPAs
Show answer and example
Answer: Clean actions supervise future dynamics for state queries but are unavailable to the deployed action policy. Indirect access through shared attention would let the noisy branch copy targets. Separate parameters alone do not prevent leakage; enforce visibility across layers.
Illustrative example: A provided “push left” label can train prediction but must not be read by the action generator and counted as a decision.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Implementation details
Why refine actions with a proximity penalty but select by the original goal cost?
LeWAM: End-to-End World and Action Modeling with JEPAs
Show answer and example
Answer: Adam penalizes deviations from the sampled plan to constrain refinement. Final selection compares unpenalized goal cost, retaining original samples in case refinement worsens them. Planning updates actions, not model weights, and does not guarantee real success.
Illustrative example: A refined action that deviates more can still win if it reaches the predicted goal more closely; keep the original if refinement fails.
Sections in the Chinese source: 核心公式, 本地复现, 数据协议、结果与边界. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Method comparison
How does action flow differ from wide Gaussian random search?
LeWAM: End-to-End World and Action Modeling with JEPAs
Show answer and example
Answer: Flow matching learns velocities from noise to action data, proposing candidates nearer the learned distribution before planning in latent dynamics. Wide Gaussian search relies on unstructured coverage. Actual task advantage requires controlled validation.
Illustrative example: A learned proposal might favor plausible pushes along a table instead of many trajectories through it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Basics
Why link predicting consequences with producing usable actions in LeWAM training?
LeWAM: End-to-End World and Action Modeling with JEPAs
Show answer and example
Answer: A world model can predict an action's future without finding goal-reaching actions efficiently. A separate policy may not shape visual representations. Joint state, dynamics, and stochastic action learning shares signals, but does not remove isolation or execution requirements.
Illustrative example: Knowing a push moves an object is not the same as deciding which direction and distance to push.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Structured Decisions / Decisions on Demand: Selective Reasoning, Transfer, and Feedback
Core mechanism
Why not combine vendor and local Jev capability numbers into one leaderboard?
Show answer and example
Answer: Different prompts, backends and data protocols confound capability. Fix public prompts, candidates and one decision; join labels only in scoring, recording language and missing coverage. This is an audit, not training, and one task class cannot represent all reasoning.
Illustrative example: Example: Vendor math under another prompt and local knowledge under a fixed interface can be listed separately, not used to declare an overall winner.
Sections in the Chinese source: 背景与主要改动, 数据与评测契约. The example illustrates the mechanism.
Structured Decisions / Decisions on Demand: Selective Reasoning, Transfer, and Feedback
Implementation details
Why submit question and candidates once in the Jev capability evaluation?
Show answer and example
Answer: One public decision prevents answer leakage or retry assistance. The scorer reads gold only afterward. Record vendor protocols separately and disclose missing items rather than silently removing them from the denominator.
Illustrative example: Example: Strip the answer field before calling the predictor; score through a separate stage.
Sections in the Chinese source: 背景与主要改动, 数据与评测契约. The example illustrates the mechanism.
Structured Decisions / Decisions on Demand: Selective Reasoning, Transfer, and Feedback
Method comparison
Do strong knowledge and weak math scores prove the model cannot reason?
Show answer and example
Answer: No. Evidence is limited to tested tasks, prompts and choice interface, possibly with language/coverage effects. Report measured tasks and gaps separately instead of generalizing to every setting.
Illustrative example: Example: Failure on one arithmetic format does not establish failure in tool planning or all mathematics.
Sections in the Chinese source: 背景与主要改动, 数据与评测契约. The example illustrates the mechanism.
Structured Decisions / Decisions on Demand: Selective Reasoning, Transfer, and Feedback
Basics
Why is the Jev capability paper an evaluation rather than a new architecture?
Show answer and example
Answer: It tests knowledge, reasoning and multilingual behavior under finite candidates. The contribution is protocol and capability boundaries, not a new training objective. An evaluator neither implements nor improves the model.
Illustrative example: Example: Converting public questions to four choices and scoring them measures performance; it is not a reasoning module.
Sections in the Chinese source: 背景与主要改动, 数据与评测契约. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Core mechanism
How does elastic routing preserve expected compute with a random expert count?
Smoothing the Top-k Exposure Boundary for Sparse Mixture-of-Experts
Show answer and example
Answer: A legal symmetric support centered at k balances more and fewer experts, keeping expectation k. Boundary experts get real sparse exposure. Deployment fixes k; training load counts actual dispatch, not fixed row length.
Illustrative example: Equal probabilities for one, two, or three experts average two, but selecting three really costs more than one.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Implementation details
Why not truncate one side of the discrete Gaussian arbitrarily?
Smoothing the Top-k Exposure Boundary for Sparse Mixture-of-Experts
Show answer and example
Answer: Keep support symmetric around k and legal to preserve E[r]=k. Execute only selected experts, count actual loads, and fix k at inference.
Illustrative example: Near the upper limit, deleting only the high side lowers mean budget.
Sections in the Chinese source: 核心公式, 本地复现. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Method comparison
How does elastic budget perturbation differ from raising fixed k?
Smoothing the Top-k Exposure Boundary for Sparse Mixture-of-Experts
Show answer and example
Answer: Raising k always spends more. Symmetric perturbation preserves expected k while changing exposure. Match parameters, initialization, and average dispatch before comparing.
Illustrative example: Equal chances of two and four experts average three, not four every time.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Basics
What is the fixed top-k exposure problem?
Smoothing the Top-k Exposure Boundary for Sparse Mixture-of-Experts
Show answer and example
Answer: Experts just below rank k rarely receive the actual token computation and training. Elastic budgets give near-boundary experts chances before fixed-k deployment.
Illustrative example: At k=2, a frequently third-ranked expert can be useful but rarely trained on those tokens.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Core mechanism
Why refine with the same noise rather than interpreting iteration index as flow time?
Few-Step Generation via Data-Space Iteration
Show answer and example
Answer: The shared generator repeatedly improves one data-space prediction from fixed original noise. DMD uses frozen real teacher and fake distribution model. Earlier refinements are detached and only the current one trains; it is not real-image regression. Local tiny runs do not replace paper metrics.
Illustrative example: Refine outline then detail from the same noise; changing noise or time semantics changes the mechanism.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
How isolate prefix, fake model, and generator gradients?
Few-Step Generation via Data-Space Iteration
Show answer and example
Answer: Roll out prefix without a graph and train the final refinement. Train fake on detached samples; freeze teacher/fake gradients during generator updates and detach the direction.
Illustrative example: Updating step four does not backpropagate through the first three steps or teacher.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Method comparison
Is direct real-image regression equivalent to DMD?
Few-Step Generation via Data-Space Iteration
Show answer and example
Answer: No. DMD uses a frozen real teacher and a fake model fitted to generated data to supply a distribution-matching direction. Per-step paired image regression replaces it.
Illustrative example: A teacher-minus-fake direction on a generated sample is not assigning a fixed real image to noise.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
What changes across Data-Space Iteration refinements?
Few-Step Generation via Data-Space Iteration
Show answer and example
Answer: A shared network updates data predictions from fixed noise. Iteration counts refinement, not the separate DMD noising time τ.
Illustrative example: Refinement three is another prediction update, not diffusion time three.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Core mechanism
How does CoTS preserve the sampled discrete trajectory under soft mixing?
CoTS: Competitive Token Selection for Latent Reasoning in Large Language Models
Show answer and example
Answer: Top-two competition sets the mixing fraction, capped at half, so the sampled embedding remains at least half even outside the soft set. End-of-reasoning disables soft mixing permanently; format guards can affect one step.
Illustrative example: Probabilities 0.6 and 0.4 give base mix 0.4 and keep 0.6 discrete; a larger gap weakens mixing.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Implementation details
Can soft mixing restart after the end-of-reasoning marker?
CoTS: Competitive Token Selection for Latent Reasoning in Large Language Models
Show answer and example
Answer: No. Per-example phase-lock is irreversible starting at the step emitting </think>. One-step format protection differs. Keep the discrete anchor even when it falls outside top-k.
Illustrative example: Later close probabilities do not reopen mixing after a sample finishes reasoning.
Sections in the Chinese source: 核心公式, 本地复现. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Method comparison
How does CoTS differ from averaging top-k embeddings?
CoTS: Competitive Token Selection for Latent Reasoning in Large Language Models
Show answer and example
Answer: Form a probability-weighted soft vector and mix it with the discrete anchor using top-two-dependent α. An unweighted mean ignores probabilities and anchoring; phase-lock controls eligibility.
Illustrative example: 0.9/0.1 and 0.5/0.5 distributions should not produce the same uniform representation.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Basics
Why keep the sampled token rather than replace it with a soft token?
CoTS: Competitive Token Selection for Latent Reasoning in Large Language Models
Show answer and example
Answer: Soft embeddings express alternatives but may drift from the sampled trajectory. The discrete anchor keeps at least half the weight while top-two competition controls the remainder.
Illustrative example: Close candidates raise soft mixing without removing the sampled token.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Core mechanism
Why control second-moment rounding in preconditioner space in 4-bit AdamW?
Rounding in Preconditioner Space: Redesigning 4-bit AdamW Optimizer-State Quantization
Show answer and example
Answer: Bias correction, square root, and reciprocal nonlinearly transform moment error into update error. ZIP-SR uses preconditioner values for stochastic rounding; ZE-EDEN handles code points and scales. Error persists through state recurrence. Use working moments for the update before packing.
Illustrative example: Small moment differences can greatly change reciprocal-root steps, so raw moment MSE misses the effective gradient scale.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Implementation details
Should current working moments be quantized before updating parameters?
Rounding in Preconditioner Space: Redesigning 4-bit AdamW Optimizer-State Quantization
Show answer and example
Answer: No. Decode old state, form new working moments, update using their unquantized values, then pack for persistence. Reversing the order injects error into the current step.
Illustrative example: Use v to precondition this step before compressing it for the next.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Method comparison
How do ZIP-SR and ZE-EDEN store second moments differently?
Rounding in Preconditioner Space: Redesigning 4-bit AdamW Optimizer-State Quantization
Show answer and example
Answer: ZIP-SR stochastically rounds in preconditioner space. ZE-EDEN excludes zero code points and calibrates block scales. Compare training loss and actual persistent bytes, not a generic four-bit rounding table.
Illustrative example: Randomly choosing adjacent codes differs from rescaling blocks to avoid zero codes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Basics
Why does state quantization affect later steps?
Rounding in Preconditioner Space: Redesigning 4-bit AdamW Optimizer-State Quantization
Show answer and example
Answer: First and second moments recur, so stored errors enter optimization after decoding at the next step. Control their effects on preconditioning and future updates, not merely bytes.
Illustrative example: Repeated downward rounding of v can inflate effective learning rates across steps.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
Why check likelihood changes for all later original actions when deleting a turn's reasoning?
When Should Agents Think? Adaptive Reasoning via Cross-Turn Estimation
Show answer and example
Answer: An unchanged immediate action does not show earlier reasoning is irrelevant to later planning. After tentative deletion, evaluate later actions in the affected context and proceed only if changes stay within tolerance. Accepted deletions persist for later compression. Checking only the current turn misses long-range dependencies, and removed token counts are not task success.
Illustrative example: Reasoning may record a safety restriction while the current action only reads a file. A later deletion relies on that restriction, so a current-turn-only check can wrongly remove critical reasoning.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
How do real trajectory requirements and the local bounded-calculator run limit conclusions?
When Should Agents Think? Adaptive Reasoning via Cross-Turn Estimation
Show answer and example
Answer: Real training data need an environment, revision, split, and reasoning/action/observation records. QA rows are not tool trajectories. The local calculator actually samples actions and runs SFT/on-policy updates, but is L1 mechanism evidence only. A few successes under one seed do not replicate ScienceWorld or four-environment RL.
Illustrative example: Calling two real calculation tools demonstrates interactive training, not long-horizon web or embodied capability.
Sections in the Chinese source: 核心公式, 本地复现, 验证与边界. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
How does a reasoning loss mask differ from actually deleting training context?
When Should Agents Think? Adaptive Reasoning via Cross-Turn Estimation
Show answer and example
Answer: A mask removes supervision for deemed-removable tokens, while other actions train under prescribed conditions. Deleting context directly changes later prediction distributions. RACE also supervises early closing and keeps denominators for all original policy tokens and turns; rescaling by amount removed would silently change the objective.
Illustrative example: If half the reasoning supervision is removed, still normalize by original token count to avoid inadvertently doubling the weight of remaining actions.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why does RACE remove reasoning that does not help an agent's trajectory?
When Should Agents Think? Adaptive Reasoning via Cross-Turn Estimation
Show answer and example
Answer: Long-horizon agents generate much reasoning that repeats information or does not affect later actions, yet adds training and inference load. RACE reduces redundancy while preserving behavioral support. Shorter is not inherently better: deletion can remove needed context, so check downstream effects rather than using word count alone.
Illustrative example: Repeating a goal without changing later choices may be redundant. A brief note with a critical path cannot be removed just because it is short.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Core mechanism
Why can changing only option labels affect guard approval while actions and trusted policy remain fixed?
One Word Opens the Gate: The Option-Channel Attack on Typed Decision Models as Agent Guardrails
Show answer and example
Answer: Labels are model inputs and their semantics can affect typed output selection without action changes. Separate unsafe approvals from benign denials. Conditional attack rate uses prohibited examples correctly rejected on clean inputs; an empty denominator is undefined, not zero.
Illustrative example: A misleading label for 'deny' can change the model's choice for the same prohibited call without changing the actual policy.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Implementation details
What is conditional ASR when no clean example was correctly rejected?
One Word Opens the Gate: The Option-Channel Attack on Typed Decision Models as Agent Guardrails
Show answer and example
Answer: Undefined because the denominator is empty; do not report zero or one. Recompute the conditional denominator in each paired bootstrap draw. Select attack labels on calibration and measure on separate evaluation data.
Illustrative example: A model already incorrectly approving every case offers no clean rejection that could flip to approval.
Sections in the Chinese source: 核心公式, 代码映射与复现边界. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Method comparison
Why not combine fail-open and fail-closed into one safety accuracy?
One Word Opens the Gate: The Option-Channel Attack on Typed Decision Models as Agent Guardrails
Show answer and example
Answer: Fail-open admits prohibited actions; fail-closed denies permitted actions. Safety and usability effects differ. Conditional ASR applies only to clean correctly rejected cases and cannot replace overall error rates.
Illustrative example: Rejecting everything can reduce unsafe approvals while making legitimate tasks unusable.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Basics
Why can Option-Channel Attack change a guard decision by altering only labels?
One Word Opens the Gate: The Option-Channel Attack on Typed Decision Models as Agent Guardrails
Show answer and example
Answer: Typed-output names also enter the model. Misleading labels can affect the mapping from an unchanged action to allow or deny, exposing the option-name channel while trusted policy and actions stay fixed.
Illustrative example: The meaning of allow and deny is unchanged, but caller-supplied names can induce the wrong choice.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Core mechanism
Why not rescale old prefix-node mass to 1/t whenever a new event arrives?
Show answer and example
Answer: Rescaling changes historical weights merely because the stream grows, confounding progress with normalization. Fixed prefix mass, streaming warm starts, and structural coupling preserve comparisons over time. Before intervening, increase the solver budget and recompute; irreversible-action gates cannot rely on stale frontier caches.
Illustrative example: A new log read should not automatically reduce the mass of an earlier critical edit. Otherwise a score change can come from the denominator rather than actual drift.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Implementation details
May OnTrack reset existing node mass from 1/m to 1/t at each event?
Show answer and example
Answer: No. Agent nodes receive fixed prefix mass 1/m_bar. Rescaling changes old masses and leakage conventions. Recompute the frontier with current coupling before irreversible actions. Planner references can warn but cannot automatically halt execution.
Illustrative example: One unrelated new event should not lower all prior masses solely by changing a denominator.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Method comparison
Why use different weights for OnTrack's early exploration grace and final alignment?
Show answer and example
Answer: Online new nodes receive age discount and a frontier mask to avoid premature penalties. At termination, cold-start without these adjustments to check the complete trace. Keeping exploration grace in final scores can hide unresolved drift.
Illustrative example: A temporary artifact gets early grace, but if still unused at termination it needs full review.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Basics
Why does OnTrack represent trajectories as dependency graphs rather than compare action text?
Show answer and example
Answer: Order and artifact dependencies determine deviation, and independent branches may have different valid orders. Structured unbalanced alignment allows partial prefix/reference matches. Without a reference, explicitly fall back to loop, retry, and stagnation signals.
Illustrative example: Swapping the order of two independent files should not be labeled deviation merely because indices change.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Core mechanism
Why must historical replay avoid resetting simulated state from the next real observation at every step?
Memento 3: Model-Based Recursive Self-Improvement through Reflective Rulebooks
Show answer and example
Answer: Per-step resets hide accumulation of world-model errors and can make an incorrect model appear to match history. Continuous replay initializes from the first observation only, then generates subsequent states solely through model transitions, testing whether the rulebook really explains the trajectory. Semantic fidelity is checked separately; fitting finite history does not show general correctness.
Illustrative example: If the first transition predicts the wrong door position, feeding the real screenshot at the second step erases the error. Continuous replay exposes its downstream consequences.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Implementation details
Why must new world-model code pass both semantic fidelity and historical execution checks?
Memento 3: Model-Based Recursive Self-Improvement through Reflective Rulebooks
Show answer and example
Answer: Code might fit some observations without expressing the current rulebook, or faithfully implement an incorrect rule. The semantic check verifies that the compiled model matches the rules; historical replay checks that it recreates observed transitions. They constrain representation and behavior separately. Mismatches on new actions still trigger repairs, and a GPU execution receipt does not establish general rule correctness.
Illustrative example: A rulebook specifies clockwise rotation, but the code hardcodes an output image. Passing one example by chance is not a faithful implementation of the rule.
Sections in the Chinese source: 核心公式, 本地复现, 结果与边界. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Method comparison
How does uniform sampling over complete-plan equivalence classes differ from voting by model member?
Memento 3: Model-Based Recursive Self-Improvement through Reflective Rulebooks
Show answer and example
Answer: Several models may propose exactly the same action sequence. Sampling by member gives duplicates more probability. Group nonempty complete sequences and sample their classes uniformly to emphasize behavioral diversity rather than model counts. Follow the selected plan until completion, mismatch, or replanning; do not randomly switch paths at every step.
Illustrative example: If three models propose left-left and one proposes right-up, member sampling has 3:1 odds, while the two plan classes each get one-half probability. Duplicates do not amplify a route.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Basics
Why does Memento 3 learn environmental rules for planning from black-box interaction?
Memento 3: Model-Based Recursive Self-Improvement through Reflective Rulebooks
Show answer and example
Answer: A new environment may not publish its transition rules. An agent sees actions and outcomes and can repeatedly err by guessing plans in language alone. Memento 3 organizes observations into a rulebook, compiles it into an executable model, and checks it with history and new actions so planning can occur in the model first. Learned rules are revisable hypotheses; agreement with history does not guarantee correctness in unseen states.
Illustrative example: After a button changes an object's orientation, propose a rotation rule and check other observed transitions. Revise the rule when new observations contradict it rather than ignoring the counterexample.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Core mechanism
Why remove the orchestrator's initial workflow prompt before training but retain worker role assignments?
Recursive Self-Improvement through Multi-Agent Self-Supervision
Show answer and example
Answer: The initial prompt contains the searched workflow. Retaining it directly could teach the model to rely on an external instruction rather than internalize useful collaboration. Worker roles preserve the execution structure and supervision applies to assistant tokens. Bradley–Terry ranking filters trajectories but self-ranked outcomes do not establish independent task capability.
Illustrative example: If the search prompt specifies retrieval then verification, remove that initial prompt so the model learns from successful execution, but keep the actual retriever and verifier turns.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Implementation details
What do two actual local LoRA rounds and lower validation loss establish?
Recursive Self-Improvement through Multi-Agent Self-Supervision
Show answer and example
Answer: They show the shared-checkpoint execution, proposal, ranking, and update chain ran, including real parameter training in a constrained two-worker text workflow. The data were self-generated trajectories, with one seed and few steps. There were no arbitrary workflow graphs or research-file tools, so lower loss does not establish stronger independent research capability.
Illustrative example: Round two may imitate its own validation text more easily and lower loss without ever testing whether its conclusions on unseen research tasks are correct.
Sections in the Chinese source: 核心公式, 本地复现, 验证与边界. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Method comparison
How does holding out the Kth Bradley–Terry-ranked trajectory differ from using all top trajectories for SFT?
Recursive Self-Improvement through Multi-Agent Self-Supervision
Show answer and example
Answer: After frozen-weight search, MASS independently collects trajectories and builds a BT ranking from pairwise comparisons. The first K−1 train the model and the Kth is for validation. The holdout separates training and checking targets, but the validation signal still comes from the chosen evaluator, not independent real-task success. Search traces cannot double as both training and generalization evidence.
Illustrative example: Use the top two of three ranked trajectories to train and the third to validate. If the third also contributes to gradients, the validation loss no longer measures a true holdout.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Basics
Why collect trajectories separately for frozen-weight search and subsequent SFT?
Recursive Self-Improvement through Multi-Agent Self-Supervision
Show answer and example
Answer: Search for workflows first without confounding exploration with parameter changes. Then generate and rank fresh trajectories, train on better samples, reserve one for validation, and only then update the shared model. The same model executes, proposes, and evaluates, but that does not make self-scoring unbiased. Self-generated validation loss is not independent task accuracy.
Illustrative example: A promising workflow during search still needs separately collected execution trajectories for training. An accidental high score from the same search trace is not independent validation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why does hierarchical recall return one layer at a time rather than immediately expanding all raw history?
Use and Disuse: Intent-Structured Experience Consolidation for Memory and Learning in LLM Agents
Show answer and example
Answer: Upper-level narratives give purpose and structure first, letting the agent drill down only when needed instead of refilling context each time. Original messages remain in child nodes, so abstraction is reversible. Reserve budget for the next turn and explicitly fail if protected complete exchanges still do not fit, rather than silently truncating them.
Illustrative example: Example: An upper node says login was repaired last week. Expand the log children only when checking the error, without returning every debug output.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
Do three archives in a controlled pressure probe show autonomous learning of intent closure?
Use and Disuse: Intent-Structured Experience Consolidation for Memory and Learning in LLM Agents
Show answer and example
Answer: No. Three real autonomous tasks produced no closure and zero archives. The separate controlled probe verifies pressure handling and protection only. Include request overhead and the next-turn estimate in budget. If protecting K complete exchanges and the dynamic window exceeds capacity, fail before main inference rather than truncate silently.
Illustrative example: Example: A predefined closure signal exercises Palim. It does not show that a model emits the signal on unknown tasks; report these runs separately.
Sections in the Chinese source: 核心公式, 本地复现, 验证与边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
Why does it matter that Precip triggers after the response but before the tool result?
Use and Disuse: Intent-Structured Experience Consolidation for Memory and Learning in LLM Agents
Show answer and example
Answer: The placement lets the system process closed intentions from the completed main response without assuming an observation that has not arrived. Tool exchanges must remain complete; compression cannot split an unfinished call. Trigger timing is a state contract, not interchangeable with inserting a summary anywhere.
Illustrative example: Example: After a tool call is issued but before its result arrives, do not summarize it as a successful verification. Wait until the real observation enters a later exchange.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
How do nested intentions differ from an ordinary rolling summary?
Use and Disuse: Intent-Structured Experience Consolidation for Memory and Learning in LLM Agents
Show answer and example
Answer: Nested intentions organize current goals and subgoals. Experiences are abstracted when their intent closes, whereas rolling summaries compress primarily by time and need not know whether a task finished. Hippocam keeps an original-message tree and recalls one layer at a time, retaining traceability after abstraction.
Illustrative example: Example: Dependency checks can finish within an unresolved debugging goal. Summarize the closed subgoal without archiving the whole task as complete.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Core mechanism
Why should some discarded candidates still receive counterfactual evaluation?
EvoAlloc: A Self-Evolving Resource Allocation Agent for Efficient Program Evolution
Show answer and example
Answer: If only selected programs are evaluated, the allocation policy cannot see high-quality candidates it missed and may become overly confident in its own preferences. Random counterfactual checks reveal the risk of missing a new best, and shadow policies need shared evidence. Those Full calls consume budget; do not exclude exploration or warmup to make savings look larger.
Illustrative example: A filter may discard an unfamiliar implementation that an occasional full evaluation reveals to be superior. That evidence corrects selective feedback that equates unfamiliarity with low quality.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Implementation details
Why charge warmup, the shadow union, and counterfactual Full evaluations to the budget, and what did the short GPU run not trigger?
EvoAlloc: A Self-Evolving Resource Allocation Agent for Efficient Program Evolution
Show answer and example
Answer: They actually consume resources and are not free administrative work. Shadow promotion also requires two observed disagreements. The best of four real local Full evaluations did not beat the initial program. Warmup was insufficient and no promotion occurred. Partial and the two-disagreement case were covered only by contract tests, not triggered in the real run.
Illustrative example: If a shadow policy requests a Full evaluation of a program discarded by the primary policy, charge it even if it never changes the primary champion.
Sections in the Chinese source: 核心公式, 本地复现, 验证与边界. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Method comparison
How does prioritizing missed new bests before cost differ from minimizing Full evaluations?
EvoAlloc: A Self-Evolving Resource Allocation Agent for Efficient Program Evolution
Show answer and example
Answer: Saving evaluations alone can miss the actual best candidate. Lexicographic ranking first protects search quality, then saves Full and Partial evaluations when missed-new-best counts tie. Random counterfactual checks of discarded candidates mitigate selective feedback, and shadow policies share evidence. Do not evaluate an allocator on accepted candidates alone.
Illustrative example: If A avoids ten evaluations but misses a new champion, while B pays for two more evaluations and retains the champion, A should not advance merely for being cheaper.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Basics
Why is continuing or stopping after Partial more flexible than an immediate Full/Discard decision?
EvoAlloc: A Self-Evolving Resource Allocation Agent for Efficient Program Evolution
Show answer and example
Answer: Partial evaluation offers cheaper initial evidence before paying full cost, avoiding exhaustive evaluation of clearly bad candidates and premature discarding of potential winners. The task needs a trustworthy Partial signal; do not invent a score for an environment that does not support one.
Illustrative example: A quick program constraint check followed by a lengthy performance evaluation can be useful, but Circle26's current interface has no Partial, so the local path supports only Full or Discard.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Core mechanism
Why renormalize a truncated Matryoshka embedding?
EmbeddingGemma 2: Unified and Efficient Multimodal Representations
Show answer and example
Answer: The retained prefix generally no longer has unit norm. Raw dot products would mix differences in retained energy with similarity. Renormalizing restores cosine scale. This relies on trained prefix representations, not arbitrary truncation, and pooling must exclude padding.
Illustrative example: Truncate two 768-dimensional vectors to 128 dimensions and normalize each again so larger retained norms do not confer an advantage.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Implementation details
What details matter for mean pooling, truncation, and precision?
EmbeddingGemma 2: Unified and Efficient Multimodal Representations
Show answer and example
Answer: Attention-mask pooling excludes padding, followed by prefix truncation and L2 renormalization. Queries and documents use their respective text prefixes; images use none. Official activation ranges can produce non-finite FP16 values, so the local path pins FP32 and rejects non-finite or zero vectors.
Illustrative example: Identical text padded to different lengths should yield the same masked representation. Including padding makes it length-dependent.
Sections in the Chinese source: 核心公式, 本地复现, 结果与边界. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Method comparison
How does MRL truncation differ from randomly dropping dimensions?
EmbeddingGemma 2: Unified and Efficient Multimodal Representations
Show answer and example
Answer: MRL trains designated prefixes, such as 128, 256, or 512 dimensions, to serve as representations after renormalization. Random subsets lack that training convention. Shorter prefixes save storage and retrieval cost, but quality must be reported by task and fixed size.
Illustrative example: Taking the first 128 of 768 dimensions follows the trained contract; randomly drawing 128 each time is a different compression method.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Basics
Does a shared 768-dimensional space require all four modalities at once?
EmbeddingGemma 2: Unified and Efficient Multimodal Representations
Show answer and example
Answer: No. A shared space lets all modalities use comparable vector similarities, while towers load as needed. Supported inputs and actually verified paths differ. Local official-checkpoint validation covers text and images only, not audio and video pipelines.
Illustrative example: Image-to-text search needs only the image and text paths; their validation does not establish audio retrieval.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Core mechanism
Why can a Lean proof still yield inconclusive even though SpecAgent never sees tests?
SpecGuard: Proving a Task Is Broken Before the Agent Cheats
Show answer and example
Answer: Test isolation keeps intent independent of potentially wrong or malicious requirements, and the Certifier checks conflict. A proof applies only to translations and connector expressions. If natural-language meaning cannot be aligned faithfully, kernel success does not prove the original task is corrupted; keep inconclusive.
Illustrative example: Sorting versus reverse-order requirements can be formalized as a conflict. Ambiguity over 'stable sorting' should not be settled solely by a limited proof.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Implementation details
Does a passed SpecGuard A100 receipt show real end-to-end model success?
SpecGuard: Proving a Task Is Broken Before the Agent Cheats
Show answer and example
Answer: No. All three archived small-model cases were inconclusive with zero conflict certificates and stopped at Spec. Passed verifies real CUDA execution and provenance. Handwritten Lean passes test mechanics, not model output.
Illustrative example: A working GPU command and saved failures do not establish that the model reached TestAgent and connector stages.
Sections in the Chinese source: 结果与验收边界. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Method comparison
Does a Lean-kernel conflict certificate prove that the original natural-language task must conflict?
SpecGuard: Proving a Task Is Broken Before the Agent Cheats
Show answer and example
Answer: It establishes no common implementation for the generated specification and test constraints, not faithful translation. Semantic connection or intent extraction may be wrong. Allow inconclusive when expression or alignment fails.
Illustrative example: A proof obtained after translating 'add one' as 'add two' does not validate the original task.
Sections in the Chinese source: 核心公式, 结果与验收边界. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Basics
Why doesn't SpecGuard let SpecAgent see tests?
SpecGuard: Proving a Task Is Broken Before the Agent Cheats
Show answer and example
Answer: Tests may conflict with trusted task descriptions; copying them into intent would prove only the tests' own requirements. SpecAgent sees description and code, TestAgent translates independently, and Lean checks conflict after semantic connection.
Illustrative example: If the description says add one but tests demand 2 become 4, do not write '2 becomes 4' into the intent specification.
Sections in the Chinese source: 背景与主要改动, 数据协议. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Core mechanism
Why refine only retained anchors without adding visual tokens?
Video Encoders Built on Image Representations
Show answer and example
Answer: Selection first preserves addressable evidence under a fixed budget. The refiner reads neighboring frames to enrich those anchors without increasing output length. Appending neighbor tokens would break the budget and confound selection with context enrichment.
Illustrative example: Ten retained patches can read neighboring motion and still output ten vectors, not ten plus 20 appended positions.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Implementation details
How are tied pairs and singletons resolved, and why can the selector fill an odd budget?
Video Encoders Built on Image Representations
Show answer and example
Answer: Add a pair only if its per-token gain is strictly higher, otherwise add a singleton. Respect quotas and remaining budget. Even with negative gains or no temporal edge, select exactly B distinct indices. Local external uniform quotas and scene proxies are explicit deviations; SITE is not implemented.
Illustrative example: One remaining slot permits only a singleton; an equal per-token gain also selects the singleton.
Sections in the Chinese source: 核心公式, 本地复现, 本地结果与验收边界. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Method comparison
What do intra-frame DPP and adjacent-frame correspondence rewards constrain?
Video Encoders Built on Image Representations
Show answer and example
Answer: The log-determinant balances quality and diversity to avoid near-duplicate tokens within frames. Cross-frame edges reward matched endpoints that preserve traceable relations. Similarity, margin, and spatial gates apply; edges are not guaranteed true trajectories.
Illustrative example: Select a cup and hand rather than two table patches; keep the cup's reliable match in the next frame to see movement.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Basics
Why select evidence from independently encoded frames before passing them to a language model?
Video Encoders Built on Image Representations
Show answer and example
Answer: Framewise encoding preserves localizable evidence but produces a large sequence for long videos. Selection balances question needs, intra-frame diversity, and cross-frame links under a budget. It does not create evidence; high motion alone need not help.
Illustrative example: A question about which object was picked up first needs that object and cross-frame relations, not every changing background patch.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Core mechanism
Why isn't one-forward analytic output gradient an exact full-model substitute?
LESSER: Post-Training Data Selection with Output-Layer Gradients
Show answer and example
Answer: The outer product is exact for an independent readout; shared bilateral projection compresses it for existing selection. Tied embeddings can introduce indirect gradients. Use validation proxies and isolate test. This is not all-layer gradient equality.
Illustrative example: Select training examples using readout gradients of validation queries, not test examples.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Implementation details
Why share random projections between LESSER candidates and queries?
LESSER: Post-Training Data Selection with Output-Layer Gradients
Show answer and example
Answer: Only shared coordinates support comparison. Project both residual and hidden sides without materializing the full outer product. Use validation query proxies, never test selection; local random tensors are diagnostics.
Illustrative example: Independent projection draws per sample destroy comparability.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Method comparison
Is LESSER fully equivalent to full-model gradient similarity?
LESSER: Post-Training Data Selection with Output-Layer Gradients
Show answer and example
Answer: No. Exactness applies to an independent output layer; tied embeddings add hidden-state paths. Projection also approximates geometry and downstream performance needs testing.
Illustrative example: Similar final-layer directions do not guarantee similar middle-layer updates.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Basics
Why use output-layer gradients as LESSER selection features?
LESSER: Post-Training Data Selection with Output-Layer Gradients
Show answer and example
Answer: Full per-example backprop is costly. Independent readout gradient follows analytically from prediction residual and final hidden state in one forward, then project for the existing selector.
Illustrative example: Keep query similarity selection but replace expensive gradient features.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Core mechanism
Why repeatedly select small-batch elites rather than pool top-K?
Show answer and example
Answer: Random minibatches followed by local top reward yield smoothed occupancy, with repetition weighting. Hard global elites are a special case and may drop useful runners-up. No same-prompt groups/critic needed, but KL remains and local tool gains aren't proven.
Illustrative example: Strongest often wins but a runner-up can win some batches; don't deduplicate then equally weight them.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Implementation details
What are FTW limits K=C and C=N?
Show answer and example
Answer: K=C keeps the whole sampled minibatch and yields uniform sampling. C=N draws the whole pool and yields global elites. Verify normalized distribution and preserve repeated selections; bandits aren't tool-world results.
Illustrative example: If K=C still favors highest rank, the implementation misses the limit.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Method comparison
Why aren't repeated minibatch elites equivalent to one global top-K?
Show answer and example
Answer: Repeated batches create soft rank occupancy; near-elites sometimes win. Global hard truncation retains only top ranks. Repeated selection counts are weights, so deduplication changes the objective.
Illustrative example: A second-tier trajectory can win a batch without the strongest opponents.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Basics
Why doesn't FTW need same-prompt groups or a critic?
Show answer and example
Answer: Stateful tool trajectories are expensive. Repeatedly draw minibatches from recent real FIFO traces, imitate within-batch elites and apply KL. Replay ranks replace some group sampling/value needs.
Illustrative example: One trace per task can still participate in a pool minibatch tournament.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Core mechanism
Why add recovery advice to the playbook only after it is verified by observable later progress?
Sentry: Learning to Recover from LLM Agent Failures at Test Time
Show answer and example
Answer: Advice proposed after failure is a hypothesis; saving it immediately can poison the playbook with bad remedies. An external verifier checks whether subsequent behavior actually recovers without reading task rewards to supply answers to the actor. Experiences are retrieved by failure class. Format errors follow hard repair, keeping mechanical correction distinct from semantic recovery.
Illustrative example: If 'refresh login' still leads to repeated errors, do not store it. Progress through subsequent steps provides evidence that can be reused for similar failures.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Implementation details
Why can't the local three-seed HotpotQA result be presented as improvement or as 24 independent tasks?
Sentry: Learning to Recover from LLM Agent Failures at Test Time
Show answer and example
Answer: Three generation seeds repeat the same eight questions. The mean was below baseline, and the extra manager and verifier calls added cost: this is a negative result. Memory is frozen after construction, holdout questions do not retune the detector, and a holdout derived from official validation is not the official test. A positive difference in one seed cannot override the overall decline.
Illustrative example: Seed 43 answers one more question correctly, but the other two perform worse. Report all paired outcomes to show instability rather than advertising only the best run.
Sections in the Chinese source: 核心规则与原文结果, 本地复现, 复现边界与失败记录. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Method comparison
What are the information trade-offs between retrieving recovery advice by failure class and keeping all experience in context?
Sentry: Learning to Recover from LLM Agent Failures at Test Time
Show answer and example
Answer: Conditional retrieval supplies relevant categories and tags only after detecting a matching failure, reducing input during normal execution. Always-on advice requires no trigger but adds irrelevant suggestions more readily. Conditional intervention depends on detection and matching: missed failures lose help, and false positives can disrupt a working policy. Less context alone does not imply more success.
Illustrative example: After a network timeout, retrieving connection recovery is more focused than carrying format and loop repair guides at each step. Misreading normal waiting as a timeout can still make the intervention harmful.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Basics
Why does Sentry specifically address stalls, repeated actions, and formatting errors during agent execution?
Sentry: Learning to Recover from LLM Agent Failures at Test Time
Show answer and example
Answer: Long-horizon agents can fail not just from missing knowledge but from loops, repeated ineffective actions, or invalid calls. Sentry detects such faults during execution and uses prior experience to support recovery. Resuming locally is not the same as ultimately answering correctly; the real local control comparison had a negative result.
Illustrative example: If an agent repeats an ineffective search, detection can trigger related advice. Even if it starts reading new pages, the final answer still needs to be checked.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Core mechanism
Why must reflection wait for forecast outcomes to mature, with memory and causal knowledge frozen during testing?
SEER: Self-Evolving Event Reasoning and Retrieval for Time Series Forecasting
Show answer and example
Answer: Future ground truth is unavailable at forecast time. Using it early to update retrieval or filtering leaks outcomes; updating memory from test errors also turns reporting into training feedback. Keeping retrieval memory, screening memory, and knowledge separate helps localize faults, but verify event time and actual availability independently.
Illustrative example: In a forecast of next week's sales, weekend news cannot be inserted into Monday's inputs. Sales learned after the test cannot be used to rewrite retrieval for that same test.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Implementation details
How do event occurrence and information availability jointly limit what a forecaster can read?
SEER: Self-Evolving Event Reasoning and Retrieval for Time Series Forecasting
Show answer and example
Answer: An event may already have occurred while reports or database entries become public later. Forecast inputs must satisfy the target-time and information-availability boundaries. The date on a story is not enough; check whether its contents were obtainable at the time. Memory can be compressed when capacity is limited, but compression cannot inject future conclusions into past contexts, and test-time reflection is forbidden.
Illustrative example: An event occurs Monday but is disclosed Wednesday. Tuesday's forecast must not use Wednesday's story, even if the headline is relabeled with Monday's date.
Sections in the Chinese source: 核心公式, 本地复现与成本判断. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Method comparison
How does fixing the forecasting model while updating retrieval and screening experience differ from fine-tuning its weights?
SEER: Self-Evolving Event Reasoning and Retrieval for Time Series Forecasting
Show answer and example
Answer: The former changes how external evidence is expanded, filtered, and organized, leaving predictor weights fixed so context evolution can be studied. Fine-tuning changes the internal mapping. The two memories and knowledge play different roles; a handwritten causal rule or new query template does not automatically teach the numerical model new dynamics.
Illustrative example: Retrieval memory can suggest checking holidays, and screening memory can exclude irrelevant news, without changing the time-series model's parameters.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Basics
Why does SEER consider available external events in addition to numerical time series?
SEER: Self-Evolving Event Reasoning and Retrieval for Time Series Forecasting
Show answer and example
Answer: Prices and demand can respond to news or policy that historical numbers alone cannot explain. SEER retrieves relevant events and improves filtering and knowledge from prior forecasting experience while keeping predictor weights fixed. Extra text can introduce noise and look-ahead risk, so check what was actually available at forecast time.
Illustrative example: An already announced holiday schedule may explain a demand shift. A sales report published afterward cannot be given to an earlier forecast, however clearly it explains the outcome.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Core mechanism
Why follow an expensive model's strategy proposal with repeated cheap execution, and assess the full best-so-far curve?
FrugalEvo: Towards Cost-Aware LLM-Guided Program Evolution
Show answer and example
Answer: Strategic planning and local code iteration require different capabilities and costs. Hierarchy reserves expensive calls for direction setting. Failures add feedback while retaining the parent program; improvement re-anchors search, and an island archive protects diversity. The best-so-far curve over the budget reflects how early useful solutions emerge, not just a lucky final score.
Illustrative example: A strong model proposes vectorization and a smaller model tests several equivalent implementations. Finding a useful speedup earlier under the same budget is more informative than one chance best at the end.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Implementation details
Why retain the parent after a generated program fails and reject fallback to bare exec?
FrugalEvo: Towards Cost-Aware LLM-Guided Program Evolution
Show answer and example
Answer: Use failure feedback for the next revision without replacing a verified parent with invalid code. Actual generated code requires Linux isolation and resource limits; fail closed if the sandbox does not start. Do not trust model-reported scores: a trusted checker verifies bounds and non-overlap. Do not dress up a negative local result by deleting failed calls or relaxing security boundaries.
Illustrative example: An invalid radius counts as a failed proposal and costs a call. Do not substitute the parent's answer and mark the candidate successful, or run it on the host when the sandbox is unavailable.
Sections in the Chinese source: 公式与原文结果, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Method comparison
How should a costly strategy model plus cheap implementer be fairly compared with weak-model-only search?
FrugalEvo: Towards Cost-Aware LLM-Guided Program Evolution
Show answer and example
Answer: Start both from the same program, budget, and call caps, but actual costs can differ when different limits bind first. Larger models are not necessarily stronger; report actual tokens, strategy trials, and refinement. The architecture does not imply collaboration always wins, and overall local results did not support that conclusion.
Illustrative example: If the weak model hits its 30-call cap with budget left and the collaboration consumes more tokens, report 'same caps,' not 'same actual cost.'
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Basics
Why does FrugalEvo want cheaper models to implement strong-model research ideas repeatedly?
FrugalEvo: Towards Cost-Aware LLM-Guided Program Evolution
Show answer and example
Answer: Using a strong model for every generation and execution step is costly, while a weaker one may lack good strategies. FrugalEvo joins stronger strategy proposals with cheaper repeated implementation to find useful programs within a budget. Evaluate actual costs and quality over the whole search, not just the final best score or a cheap individual call.
Illustrative example: A strong model suggests a direction and a weak model tries several code variants. If repeated failures still consume substantial budget, the division of labor may not pay off.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
Why connect file reads to the writers of the still-current versions in a command dependency graph?
Credit Where It Matters: Dependency-Aware Policy Optimization for Terminal Agents
Show answer and example
Answer: Reads consume the version active at that moment. Earlier overwritten writes may not affect the final artifact. Trace backward from verifier-checked outputs to connect data-contributing commands to the result and avoid sharing terminal advantage with unrelated steps. Graph construction needs trusted execution records, not guesses from command text.
Illustrative example: If a configuration is written and overwritten before a test reads it, the unused old write should not receive the same credit as the current version.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
What boundaries apply to multiple commands per step, missing records, and verifier-generated files?
Credit Where It Matters: Dependency-Aware Policy Optimization for Terminal Agents
Show answer and example
Answer: Sum command credits within one step, rather than averaging. Missing records or fully tied weights revert to unit weights. Exclude verifier-written resources and stdout values already present in the task prompt; paths and line numbers come from trusted tracing. Local work parses no shell, runs no real Qwen RL, and only verifies the mechanism on four hand-built chains.
Illustrative example: If the grader generates a result file, treating it as an agent write rewards a nonexistent contribution. Remove such artifacts from the tracer input.
Sections in the Chinese source: 公式与关键区别, 本地复现（机制验证）, 复现边界与下一步. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
How does dependency credit differ from uniform trajectory weighting in GRPO?
Credit Where It Matters: Dependency-Aware Policy Optimization for Terminal Agents
Show answer and example
Answer: Unit weights do not distinguish supporting actions, final writes, and unrelated commands. DepGPO distributes step credit by traceable writes and distance, then normalizes to preserve token-weighted mean advantage and sign. It depends on tracer accuracy and does not prove causality or individual necessity.
Illustrative example: An input read followed by an answer write can get indirect credit. Unrelated temporary-file cleanup gets less weight, while negative trajectory advantage stays negative.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why track command dependencies in DEPGPO when terminal tasks only yield final success or failure?
Credit Where It Matters: Dependency-Aware Policy Optimization for Terminal Agents
Show answer and example
Answer: A long trace contains many file reads and writes; terminal reward cannot identify which commands produced the result. DEPGPO connects verification to earlier effective work through actual dependencies. Dataflow is not causal contribution of every command, and missing records or verifier side effects require explicit boundaries.
Illustrative example: Trace the source file read by a final test to the command that wrote it. Unrelated directory browsing should not get equal credit just because it appeared in the trace.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
Why lower local credit correction when within-action return variance is high at the same anchor state?
AdaStep: Adaptive Step Credit Weighting for Agentic Reinforcement Learning
Show answer and example
Answer: Random continuations after one action can produce very different outcomes, implying the local action explains little of the variation. Variance decomposition trusts local advantage more when between-action differences dominate; otherwise it preserves global advantage and suppresses the correction. Estimate weights from raw returns rather than arbitrarily normalizing away variance relationships.
Illustrative example: If rollouts from both actions A and B produce zeros and perfect scores, continuation randomness dominates. Do not strongly attribute a single outcome to either action.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
How are few actions, few samples, zero variance, and normalized returns handled?
AdaStep: Adaptive Step Credit Weighting for Agentic Reinforcement Learning
Show answer and example
Answer: Estimate the ratio only with at least two actions, two samples per action, and positive total variance; otherwise use the paper's w=1 fallback. Equal returns give zero local advantage. Weight statistics use raw returns, and groups include task IDs to avoid mixing names across tasks. No real environment or policy training ran locally.
Illustrative example: One action cannot reveal between-action variation, but that does not justify inventing w=0. Identical rewards should not create local correction out of nothing.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
How does AdaStep's return-statistics approach differ from training a critic for step evaluation?
AdaStep: Adaptive Step Credit Weighting for Agentic Reinforcement Learning
Show answer and example
Answer: A critic learns state or action values to organize credit. AdaStep estimates local-correction reliability directly from observed returns for different actions at the same anchor, with no separate critic. It saves a predictor but still needs repeated states and sample coverage. Critic-free does not imply noiseless or universally more accurate.
Illustrative example: Repeated actions from the same webpage state supply empirical variation. A single occurrence cannot provide a reliable local estimate by itself.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why does AdaStep distinguish an action's effect from luck in later trajectories?
AdaStep: Adaptive Step Credit Weighting for Agentic Reinforcement Learning
Show answer and example
Answer: At the same intermediate state, later decisions still affect success. Assigning terminal differences directly to the first action is noisy. AdaStep uses multiple observed trajectories to assess local credit reliability, retaining global outcomes while cautiously adjusting step signals. It trains no separate critic and does not prove causal effects from few samples.
Illustrative example: After the same lookup, one continuation answers correctly and another transcribes incorrectly. Compare between-action differences with within-action fluctuations rather than blaming the lookup for all variation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Core mechanism
Why place high-priority products in ascending order near the prompt end?
RPTune: Learned Context Curation for LLM Catalog Search
Show answer and example
Answer: Learned query–product scores determine what stays; positions near the response are a separate optimization. Post-training adapts to that catalog input. Local tests check arrangement, not a universal position preference.
Illustrative example: With three slots, drop weak candidates, then place the best affordable product nearest the answer. Test selection and placement separately.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Implementation details
Why does RPTune sort retained top-k products in ascending score order?
RPTune: Learned Context Curation for LLM Catalog Search
Show answer and example
Answer: Its layout puts high scores near the end. After retention, sort ascending rather than defaulting to descending. The local rank/prune/position kernel does not call the paper's LLM or run full post-training, so it cannot report full catalog-search gains.
Illustrative example: Placing the top-scoring item first reverses the intended end preference.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Method comparison
Are RPTune's product scoring and prompt ordering both just vector top-k?
RPTune: Learned Context Curation for LLM Catalog Search
Show answer and example
Answer: No. Scoring adds learned query and product adjustments to the semantic dot product. After top-k, a prescribed positioning rule orders the LLM input. Vector top-k alone has neither learned adjustment nor that order. Ablate set selection and positioning separately.
Illustrative example: Two methods can keep the same ten products in different orders, or choose different sets under the same ordering rule.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Basics
What does RPTune pruning decide, and what does positioning decide?
RPTune: Learned Context Curation for LLM Catalog Search
Show answer and example
Answer: Pruning decides which catalog products enter the prompt; positioning sets the order of retained products. One changes the selectable set and the other changes context for reading the same set. Reordering cannot recover a dropped product.
Illustrative example: Keep ten products out of a thousand, then reorder the ten. The second step cannot restore deleted items.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why repair preference with cached state differences while freezing Repair-State's encoder and head?
Not All Is Lost: Repairing Lossy User Preference States of Personalization Encoders
Show answer and example
Answer: Cached states retain evidence at different times. Select long-range, recent, or burst segments and add weighted differences to restore compressed directions without retraining. It cannot recover uncached evidence and can reintroduce stale interests.
Illustrative example: Illustration: Retrieve recent baby-product activity from cache to supplement a profile dominated by long-term movie viewing.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why check valid history before Repair-State top-k and masked softmax?
Not All Is Lost: Repairing Lossy User Preference States of Personalization Encoders
Show answer and example
Answer: Mask padding, future states, and other users before selection and normalize only selected valid states. Align differences with z. Define empty-history fallback rather than injecting random evidence. Frozen encoders do not prevent data leakage.
Illustrative example: Illustration: With two events and k=5, zero-scored padding must not beat valid negative scores and pull z toward zero.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does Repair-State differ from retraining the encoder?
Not All Is Lost: Repairing Lossy User Preference States of Personalization Encoders
Show answer and example
Answer: Postprocessing uses an existing forward cache while preserving encoder and head, reducing deployment cost. Retraining can change feature extraction. Residuals cannot recreate omitted evidence, and selection may not transfer to all tasks. The local diagnostic does not establish capability rankings.
Illustrative example: Illustration: A stored burst can be restored, but an event ignored entirely by the encoder cannot.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why aren't a historical ht and current preference z interchangeable?
Not All Is Lost: Repairing Lossy User Preference States of Personalization Encoders
Show answer and example
Answer: ht represents a contextual position and z summarizes current preference for the head. Equal dimensions do not imply the same time or coverage. Repair selects evidence while retaining current state rather than replacing it wholesale.
Illustrative example: Illustration: Last year's camera state cannot replace a current profile that includes camping.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why does GrIS use both item features and collaborative edges to construct SIDs by graph partitioning?
Show answer and example
Answer: Content proximity and co-consumption differ. Hierarchical partitions balance semantic features and collaborative links; without graph evidence fall back to content rather than inventing edges. Local partition and state checks do not verify generative or online recommendation quality.
Illustrative example: Co-consumed camping gear can connect despite visual differences; new items without edges must mainly use content.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
Why save node identities and parent paths at each GrIS recursive split?
Show answer and example
Answer: Refinement stays inside each current subgraph. Without parent paths, local child IDs from distinct branches can collide. Implement smoothing, semantic combination, and empty-graph handling explicitly; integer outputs alone do not verify content-and-collaboration hierarchy.
Illustrative example: Two parent clusters can each have child 2, but their full SIDs must differ.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
How does GrIS's content fallback for an empty graph differ from inventing adjacency?
Show answer and example
Answer: Fallback openly uses only available content and gives a defined control. Synthetic edges presented as observations change the objective. Report graph availability and do not infer large-scale ranking from diagnostics.
Illustrative example: A new product can use title semantics, not fabricated links to popular items claimed as collaborative evidence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why formulate GrIS SID construction as hierarchical graph partitioning?
Show answer and example
Answer: Nodes supply content and edges supply collaborative consumption relations. Recursive partitioning creates coarse-to-fine paths beyond content reconstruction. Exposure and popularity affect graph structure, so hierarchy is not a guaranteed true taxonomy.
Illustrative example: Cameras and lenses can share a high-level interest region through co-consumption and separate into lower-level content groups.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Basics
Why can effective training time be low despite saturated GPUs?
Optimizing Effective Training Time for Large-Scale Recommendation Systems
Show answer and example
Answer: ETT counts wall-clock time actually training on new data. Init, compilation, save, publishing, and recovery extend the denominator. Kernel improvements do not fix all pauses. Local lifecycle checks do not estimate production gains.
Illustrative example: Twenty new-data minutes in an hour is about one-third even if GPUs saturate during those minutes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Core mechanism
How does stage accounting choose between optimizing compilation, recovery, and kernels?
Optimizing Effective Training Time for Large-Scale Recommendation Systems
Show answer and example
Answer: Separate new-data training from lifecycle losses, then identify avoidable time and owners. If training occupies little of the run, reducing interruptions can matter more than faster operators.
Illustrative example: Cutting one recovery in a run with ten training minutes and 30 recovery minutes can beat a 10% kernel speedup.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Implementation details
Why define overlap and recovery replay in ETT?
Optimizing Effective Training Time for Large-Scale Recommendation Systems
Show answer and example
Answer: Use actual wall time and count only training on new data. Do not double-count asynchronous save overlap or count replayed old batches as progress. Attribution can overlap without changing ratio accounting.
Illustrative example: Five overlapping minutes do not make an hour 65 minutes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Method comparison
Why can ETT, stable samples/s, and MFU suggest different priorities?
Optimizing Effective Training Time for Large-Scale Recommendation Systems
Show answer and example
Answer: Throughput measures training speed; MFU measures compute utilization; ETT measures new-data share over the full lifecycle. More failures or compilation can erase faster kernels.
Illustrative example: Steps can be 20% faster while frequent restarts reduce daily progress.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why ask neighboring agents only when private evidence is insufficient?
AgentWebRec: Compact Evidence Fusion over the Agent Web for Personalized Recommendation
Show answer and example
Answer: Personal relevant memories are primary. Retrieve with semantics and recency; low confidence triggers compact neighbor patterns rather than raw history. Local gate/fusion checks are not full agent capability.
Illustrative example: Use repeated vegetarian preferences directly; seek general travel patterns for vague new interest without sharing itineraries.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
What should time decay and collaboration thresholds test separately?
AgentWebRec: Compact Evidence Fusion over the Agent Web for Personalized Recommendation
Show answer and example
Answer: Keep time units consistent and avoid giving future events anomalous gains. Compare confidence κ to its own threshold, separate from retrieval rank. Check sorting, expiry, and trigger boundaries.
Illustrative example: A top-scoring memory is not automatically sufficient evidence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
What is the trade-off versus always aggregating neighbors?
AgentWebRec: Compact Evidence Fusion over the Agent Web for Personalized Recommendation
Show answer and example
Answer: Always collaborating may cover more but adds communication and noise. Confidence gating saves calls; overconfidence misses help, underconfidence calls too often. Local results are diagnostic.
Illustrative example: Do not inject popular gaming patterns into a request with clear photography evidence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
What do private memory and neighbor patterns each provide?
AgentWebRec: Compact Evidence Fusion over the Agent Web for Personalized Recommendation
Show answer and example
Answer: Private memory is personal evidence retrieved by meaning and recency. Neighbor patterns supplement weak coverage, without overriding strong direct evidence or becoming facts about the target.
Illustrative example: Use the user's lightweight-camera preference before asking about an unfamiliar outdoor need.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Core mechanism
Why give teacher textual coordinates while student receives only image and question?
Where-OPD: Spatially Guided On-Policy Self-Distillation of MLLMs with Synthetic Scenes
Show answer and example
Answer: Exact synthetic object/location metadata gives privileged scoring on student prefixes. Deployment lacks that metadata, so keep student inputs clean. Preserve spatial mask; random-logit tests don't prove spatial ability.
Illustrative example: Teacher can correct left-of-box relationships, but test students cannot be told coordinates.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Implementation details
Does local Where-OPD generate scenes and load an MLLM?
Where-OPD: Spatially Guided On-Policy Self-Distillation of MLLMs with Synthetic Scenes
Show answer and example
Answer: No. It checks teacher KL under a spatial mask aligned to generated student positions, with frozen teacher. Random logits aren't visual evidence.
Illustrative example: Two hand-built left/right logits don't constitute image reasoning.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Method comparison
How does spatial privileged text differ from ordinary text distillation?
Where-OPD: Spatially Guided On-Policy Self-Distillation of MLLMs with Synthetic Scenes
Show answer and example
Answer: The method needs matching actual scene images and spatial annotations. Random text distributions don't verify visual integration; privilege doesn't prove student reliance on images.
Illustrative example: A correct description paired with the wrong image still yields a computable loss but meaningless guidance.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Basics
Why use synthetic scene coordinates as privileged Where-OPD input?
Where-OPD: Spatially Guided On-Policy Self-Distillation of MLLMs with Synthetic Scenes
Show answer and example
Answer: Generated scenes provide exact objects/positions for teacher while student sees image/question. Internalize spatial hints without coordinate access at deployment.
Illustrative example: Teacher knows positions; student learns left/right from pixels.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Core mechanism
Why interpret high VLM-reward gradient as sensitivity rather than correct credit?
Token-Level Video Reinforcement Learning
Show answer and example
Answer: It measures local response of the current reward function. Mean-normalized token reweighting controls scale, but shortcut-driven sensitivities need not be causal. Local checks neither train videos nor prove perception gains.
Illustrative example: A corner watermark can receive a large gradient without being a route to better video quality.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Implementation details
Does local TVRL train a video diffusion model?
Token-Level Video Reinforcement Learning
Show answer and example
Answer: No. It computes gradient-based credits and mean normalization. Real video, frozen VLM gradients and policy training remain; handle all-zero norms explicitly. Random inputs are not video capability.
Illustrative example: Hand-supplied gradient sizes validate weighting without generating videos.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Method comparison
How does TVRL sensitivity differ from uniformly broadcasting advantage?
Token-Level Video Reinforcement Learning
Show answer and example
Answer: Normalized gradient norms emphasize sensitive positions while uniform credits ignore them. Quality depends on the frozen VLM; gradients alone are not causal evidence.
Illustrative example: Evaluator sensitivity to a word can be a shortcut.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Basics
Why use gradient sensitivity for video-token credit in TVRL?
Token-Level Video Reinforcement Learning
Show answer and example
Answer: Terminal video reward is coarse. Sensitivity through a frozen evaluator localizes group advantage. Sensitivity means response to perturbations, not verified correctness.
Illustrative example: Action terms may affect video reward more than connective words.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
Why measure fixed-budget task coverage even when pass@1 improves?
Sharpening Tax in Post-Training
Show answer and example
Answer: Sharper distributions hit familiar solutions more often but may lose rare solutions accessible with several draws. Compare base/post coverage under equal budget and adapt temperature by difficulty. First-hit accuracy, entropy and task coverage are not interchangeable.
Illustrative example: Better common-item answers can hide lost rare correct solutions.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
Why hold K and task inputs fixed when computing Tax_K?
Sharpening Tax in Post-Training
Show answer and example
Answer: Coverage changes with samples and tasks. Match both and retain the sign of base minus post-training coverage. Local code is a metric only.
Illustrative example: Many base samples against one post-training sample confound the estimated tax.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
How does posterior-conditioned temperature differ from uniformly raising temperature?
Sharpening Tax in Post-Training
Show answer and example
Answer: Allocate temperature by estimated difficulty to balance exploitation and coverage. Global heating treats all tasks alike. Bad estimates may misallocate, so recovery isn't guaranteed.
Illustrative example: Mastered problems may need little exploration while difficult problems benefit from branching.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
Why measure both first-attempt success and fixed-budget coverage in Sharpening Tax?
Sharpening Tax in Post-Training
Show answer and example
Answer: Training may raise probability of familiar successes while making other solutions or tasks unreachable. Fixed-budget coverage exposes this tradeoff.
Illustrative example: One problem becomes usually correct while another never yields a correct sample, reducing coverage.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
Why penalize relative length only if all answers are long and posteriors concentrated?
RLCPR: Rethinking Probability-Based Reinforcement Learning From Posterior Concentration
Show answer and example
Answer: Similar posteriors indicate limited distinction from reasoning length; widespread long replies support compressing redundant ones. Penalize relatively only after both gates. Equality and quantile rules matter; real entropy/posteriors must come from a model.
Illustrative example: Favor shorter among equally supported long replies, not a long reply that clearly increases answer support.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
Do exact length or posterior thresholds activate RLCPR?
RLCPR: Rethinking Probability-Based Reinforcement Learning From Posterior Concentration
Show answer and example
Answer: No. Minimum length must strictly exceed its threshold and posterior range must be strictly below its bound. Entropy CDF also uses strict less-than on ties. Handcrafted numbers test formulas, not RL gains.
Illustrative example: Minimum length exactly at threshold is not penalized.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
Are RLCPR entropy strata and length gating the same choice?
RLCPR: Rethinking Probability-Based Reinforcement Learning From Posterior Concentration
Show answer and example
Answer: Entropy strata shape examples before sampling. Length/posterior gates adjust rewards inside generated groups. Different inputs and stages, not one global length threshold.
Illustrative example: Deciding which tasks to sample precedes checking whether current replies are all redundant.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
Why not apply a uniform length penalty to all long replies in RLCPR?
RLCPR: Rethinking Probability-Based Reinforcement Learning From Posterior Concentration
Show answer and example
Answer: Long reasoning can be necessary. Enable relative penalty only when all group replies are long and answer posteriors concentrated, where length may add little distinction.
Illustrative example: Repeated explanation after agreement differs from exploring uncertain critical branches.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
Why not rank replies with overlapping reward intervals?
Range-GRPO: Policy Optimization via Pairwise Relations among Reward Intervals
Show answer and example
Answer: Small point differences may be judge noise. Use determinate calibrated interval relations and aggregate relative group advantage without inventing directions. Degenerate intervals recover point rewards. The kernel is not judge calibration.
Illustrative example: Do not favor one broad overlapping interval merely because its center is higher.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
Does local Range-GRPO calibrate a conformal judge?
Range-GRPO: Policy Optimization via Pairwise Relations among Reward Intervals
Show answer and example
Answer: No. It consumes intervals and tests comparisons and the point limit. Coverage needs real calibration data/judging. Degenerate intervals should recover the DR-GRPO limit, not arbitrary smoothing.
Illustrative example: A wide interval passing an interface test doesn't establish coverage.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
How does treating overlap as uncertain differ from midpoint ranking?
Range-GRPO: Policy Optimization via Pairwise Relations among Reward Intervals
Show answer and example
Answer: Midpoints always impose order even inside large uncertainty. Interval comparison discards dubious direction, at the cost of sparser supervision. It does not prove faster/better training.
Illustrative example: Different midpoints with overlap needn't imply a clear preference.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
Why use reward intervals instead of point scores in Range-GRPO?
Range-GRPO: Policy Optimization via Pairwise Relations among Reward Intervals
Show answer and example
Answer: A point hides judge uncertainty. Intervals identify pair orderings that can be justified and avoid treating tiny noise as preference.
Illustrative example: Strongly overlapping score ranges need not produce a winner.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Core mechanism
Why verify diagnostic claims before error pricing?
My FAULT: Self-Diagnosis as Credit Assignment in Self-Evolving Agentic Reinforcement Learning
Show answer and example
Answer: Hindsight explanations can rationalize. Evidence-verified location and class drive relative online prices and conserved terminal credit redistribution. No invented success reward or jointly trained local diagnosis model.
Illustrative example: If search was correct but summary dropped a fact, don't penalize search merely because the model blames it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Implementation details
What conservation invariant should My FAULT check?
My FAULT: Self-Diagnosis as Credit Assignment in Self-Evolving Agentic Reinforcement Learning
Show answer and example
Answer: Valid-step credits sum to original return. Mean cost and step count use the same mask. Local screening/cost/conservation does not jointly train a policy and diagnosis model.
Illustrative example: Including padding in mean cost breaks conservation on valid steps.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Method comparison
How does conserved redistribution differ from an extra error penalty?
My FAULT: Self-Diagnosis as Credit Assignment in Self-Evolving Agentic Reinforcement Learning
Show answer and example
Answer: Conservation preserves total terminal return and changes only allocation. Extra penalties change the objective and incentives.
Illustrative example: Lower one step's credit and offset elsewhere rather than deduct from the entire return twice.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Basics
Why verify My FAULT self-diagnoses before pricing errors?
My FAULT: Self-Diagnosis as Credit Assignment in Self-Evolving Agentic Reinforcement Learning
Show answer and example
Answer: Self-explanations can invent causes. Only evidence-supported categories/locations enter online pricing; relative costs allocate terminal credit.
Illustrative example: A claim that the tool failed needs trace evidence before penalizing tool steps.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
Why rescale an entire group instead of clipping trajectories one by one?
Asynchronous LLM Post-Training: Group-Mass Capping and Convergence Analysis
Show answer and example
Answer: Independent clipping changes relative weights. A shared factor limits total ratio mass and retains relative contributions. Sampling and moment assumptions remain; a formula test is not an asynchronous training system.
Illustrative example: Uniformly scale stale trajectories when total mass is too large; separate caps can flatten differences.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
How do GMC weights behave below the total cap?
Asynchronous LLM Post-Training: Group-Mass Capping and Convergence Analysis
Show answer and example
Answer: Use multiplier one until total exceeds cap; then multiply by cap divided by sum. Check nonnegative weights and group boundaries. Local kernel does not measure asynchronous cluster throughput.
Illustrative example: Do not pool two prompts: a large weight in one should not scale the other.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
What does a group cap preserve that per-trajectory clipping may not?
Asynchronous LLM Post-Training: Group-Mass Capping and Convergence Analysis
Show answer and example
Answer: Common scaling retains ratios between weights. Independent caps change them. Theory still requires conditions such as common second moments; it does not repair arbitrary staleness.
Illustrative example: Clipping may make large and medium weights equal, while group scaling preserves their ratio.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
Why cap total group importance mass in GMC-GRPO?
Asynchronous LLM Post-Training: Group-Mass Capping and Convergence Analysis
Show answer and example
Answer: Asynchronous stale policies can produce huge ratios. Apply one multiplier to the group to cap mass while preserving within-group proportions, unlike independent clipping.
Illustrative example: Scale the whole inflated group instead of cutting only its largest member.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Core mechanism
Why penalize rejected-response tokens less if they support the preferred direction?
GAW-PO: Preference Optimization with Gradient-Aligned Token Weights
Show answer and example
Answer: A whole response's rejection does not condemn all tokens. Smaller negative weights for aligned gradients avoid suppressing shared useful steps. This changes credit, not factual verification.
Illustrative example: If both openings are right and only the rejected conclusion is wrong, an aligned opening needn't receive the same penalty.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Implementation details
How does GAW-PO's positive-part cosine handle aligned, orthogonal and opposite gradients?
GAW-PO: Preference Optimization with Gradient-Aligned Token Weights
Show answer and example
Answer: Weight is one minus positive cosine. Perfect alignment approaches zero; orthogonal/opposite directions keep weight one; partial positive alignment reduces penalty. Actual gradients are required. Local evidence is only the weight kernel.
Illustrative example: A negative cosine is first set to zero, so the weight does not exceed one.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Method comparison
How does GAW-PO gradient alignment differ from weighting by token likelihood?
GAW-PO: Preference Optimization with Gradient-Aligned Token Weights
Show answer and example
Answer: Likelihood measures how often the current model produces a token. Alignment measures whether penalizing it conflicts with preferred updates. Common tokens can be irrelevant and rare tokens helpful; this is a mechanism comparison, not an empirical ranking.
Illustrative example: Do not punish a rare term merely for low probability if it supports a correct explanation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Basics
Does GAW-PO label complete replies or each token as preferred/rejected?
GAW-PO: Preference Optimization with Gradient-Aligned Token Weights
Show answer and example
Answer: Labels compare two replies to one prompt; there is no per-token annotation. Rejected replies can contain good steps or shared wording. Gradient alignment supplies credit at a finer granularity, not additional human token truth.
Illustrative example: If both answers start with the correct equation and differ only at the end, the rejection does not label the shared equation wrong.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Core mechanism
Why decay nearby credit rather than reset every completed step?
Dependency-Aware Reward Shaping for Agentic Reinforcement Learning
Show answer and example
Answer: Invalid prerequisites affect dependent branches, not independent work. Graph-distance decay and potential differences encode breakage/repair. Valid predicates and edges are prerequisites; local transitions aren't agent gains.
Illustrative example: Ticket failure weakens hotel credit but not independent reimbursement materials.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Implementation details
Does DARS repair change only immediate reward?
Dependency-Aware Reward Shaping for Agentic Reinforcement Learning
Show answer and example
Answer: No. Update dependency state/potential and derive reward from differences. Otherwise later credit uses stale breakage. Local state kernel doesn't train paper model; upstream interfaces aren't released training code.
Illustrative example: Repaired flights restore downstream state rather than add an isolated bonus.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Method comparison
How does dependency-distance decay differ from global failure penalty?
Dependency-Aware Reward Shaping for Agentic Reinforcement Learning
Show answer and example
Answer: Decay depends on relation to the broken prerequisite; global penalty ignores it. Wrong graph or predicates still misassign credit, so validate them.
Illustrative example: Flight-dependent hotels are affected, not unrelated account verification.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Basics
Why represent DARS progress as a dependency predicate graph?
Dependency-Aware Reward Shaping for Agentic Reinforcement Learning
Show answer and example
Answer: Completed steps can rely on later-invalid conditions. Graph validation, invalidation and repair update related credit without clearing independent branches.
Illustrative example: Canceling a booking can affect itinerary but not unrelated document preparation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Core mechanism
Why take absolute log ratios before response averaging?
CARM: Cancellation-Aware Response Masking for LLM Reinforcement Learning
Show answer and example
Answer: Opposing probability shifts can cancel in a normal sequence mean. Absolute averaging detects magnitude and masks large mismatch. It measures policy shift, not answer quality, with a detached gate.
Illustrative example: Opposite shifts across sentence halves produce near-zero total signed drift but large mean absolute drift.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Implementation details
Should CARM differentiate through the response gate?
CARM: Cancellation-Aware Response Masking for LLM Reinforcement Learning
Show answer and example
Answer: No. Freeze selection, measure valid response tokens only and apply the specified threshold. No Qwen3.5 training or measured stability improvement.
Illustrative example: Don't add gradients through a threshold recomputed from updated probabilities.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Method comparison
How does CARM differ from filtering on geometric ratio near one?
CARM: Cancellation-Aware Response Masking for LLM Reinforcement Learning
Show answer and example
Answer: Geometric ratios allow signed cancellation; CARM measures unsigned change and can reject internally shifted responses near ratio one. This is not quality scoring.
Illustrative example: A balanced ledger can still hide large cash movements.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Basics
Why take absolute token log ratios in CARM?
CARM: Cancellation-Aware Response Masking for LLM Reinforcement Learning
Show answer and example
Answer: Signed shifts can cancel across positions and conceal severe drift. Average absolute magnitude and use a response-level gate to detect cancellation risk.
Illustrative example: A sharp rise at one token and equal fall at another can average to zero.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Core mechanism
Why merge within frames before removing cross-frame redundancy?
VETO: Video Efficient Token Optimization for Vision Language Models
Show answer and example
Answer: Intra-frame merging shrinks per-frame candidates, lowering temporal matching cost. Matching all raw patches first already pays that search cost and can yield different correspondences. The order is part of the mechanism.
Illustrative example: Compress 100 patches per frame to 20 before matching adjacent frames; the reverse does not save the same work.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Implementation details
What does the local farthest-point representative test preserve, and what does it omit?
VETO: Video Efficient Token Optimization for Vision Language Models
Show answer and example
Answer: Deterministic farthest-point representatives test the space-first, time-second sequence and budget, and execute compression on GPU. There is no paper 7B VLM or real video benchmark. The heuristic does not reproduce semantic merging, and author throughput is not a local result.
Illustrative example: Meeting the token cap confirms both stages ran, not that small-object movement remains recognizable.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Method comparison
At fixed token budget, what do more frames versus more positions per frame preserve?
VETO: Video Efficient Token Optimization for Vision Language Models
Show answer and example
Answer: More frames favor temporal coverage; more positions favor spatial detail. The local tensor is frames × positions × dimensions, with each keep count bounded by its source axis and output position count equal to their product. No allocation is universally proven optimal.
Illustrative example: Four frames × 16 positions and 16 frames × four both use 64 tokens; the former sees more detail, the latter may catch brief events.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Basics
Why are there both intra-frame and cross-frame redundancies in video?
VETO: Video Efficient Token Optimization for Vision Language Models
Show answer and example
Answer: Nearby patches can depict one background, and adjacent frames can repeat one scene. VETO removes duplication along space and time. Each compression axis risks losing different details, so total rate alone does not show required motion or local cues survived.
Illustrative example: A wall repeats within and across frames, but a fleeting hand movement needs both temporal coverage and spatial detail.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Core mechanism
Why train TESS on loss-gap pseudolabels instead of directly optimizing selection?
Show answer and example
Answer: Compare train-only and validation-guided models from the same initialization per example, then learn a transferable pointwise selector. Separating valuation from prediction avoids some degenerate direct objectives. Validation guidance is not test data.
Illustrative example: Use loss changes to learn which examples help, then predict candidate value without reading test outcomes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Implementation details
Can selector gradients flow back through Δ_i into U/W?
Show answer and example
Answer: No. Train U/W in order from identical initialization and detach their saved loss gaps. Scores may be negative; do not use them as probabilities without transformation or substitute test for validation.
Illustrative example: A negative value label should not be silently clamped to zero.
Sections in the Chinese source: 核心公式, 本地复现. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Method comparison
How do loss gaps differ from selecting the highest raw loss?
Show answer and example
Answer: A shared-init model pair estimates response to validation guidance; raw loss measures difficulty. Hard examples need not be useful.
Illustrative example: Persistent noise can have high loss without a valuable guided change.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Basics
Why make training-example value a learned prediction?
Show answer and example
Answer: Teacher loss gaps create pseudolabels, then a selector scores an independent pool. This amortizes expensive valuation rather than rerunning training across the large pool.
Illustrative example: Estimate value on a small set, then score candidates unseen by the teacher models.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Core mechanism
How is one gradient position per TACO column different from elementwise sign?
TACO: Ternary Absolute-max Column-wise One-sparse Optimizer for LLM Fine-Tuning
Show answer and example
Answer: Select the maximum absolute gradient row per column and retain only its sign. Low-precision column state saves memory. SignSGD updates all nonzero locations and is not one-sparse.
Illustrative example: [0.2,−0.9,0.4] yields only a negative update at row two rather than +,−,+.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Implementation details
What if TACO's argmax runs along the wrong axis?
TACO: Ternary Absolute-max Column-wise One-sparse Optimizer for LLM Fine-Tuning
Show answer and example
Answer: Maximize over rows separately for each column, not columns per row. Check zeroed positions and low-precision column momentum, not shape alone.
Illustrative example: A 3×5 matrix has up to one selected entry in each of five columns.
Sections in the Chinese source: 核心公式, 复现边界. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Method comparison
What differs between TACO and elementwise signSGD?
TACO: Ternary Absolute-max Column-wise One-sparse Optimizer for LLM Fine-Tuning
Show answer and example
Answer: TACO chooses one max-absolute row and keeps its sign; signSGD can update all nonzero entries. Their supports differ even though both use signs.
Illustrative example: For [1,−4,2], only TACO's second entry is −1 and the others are zero.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Basics
Why keep one position per column?
TACO: Ternary Absolute-max Column-wise One-sparse Optimizer for LLM Fine-Tuning
Show answer and example
Answer: The max absolute entry yields a one-sparse signed direction and permits low-precision column-level state instead of dense per-parameter state.
Illustrative example: A column with many nonzeros updates only its largest-magnitude entry this step.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Core mechanism
Why keep global instructions in a persistent REA prefix?
Role-aware Heuristic Episodic Attention for Conversational LLMs
Show answer and example
Answer: Relevance ranking should not discard standing constraints. REA separates them from episodic retrieval, which applies role-specific preservation, compression, or omission. Context selection is not lossless reconstruction; policies require verification.
Illustrative example: A standing Chinese-output requirement must not disappear during a movie discussion merely for low keyword similarity.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Implementation details
Does the local REA top-k path include real LLM compression?
Role-aware Heuristic Episodic Attention for Conversational LLMs
Show answer and example
Answer: No. It executes role separation and episodic retrieval without a compression LLM. Distinguish selection from generated summaries.
Illustrative example: Choosing three passages does not show an LM compressed them faithfully.
Sections in the Chinese source: 复现边界. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Method comparison
How does role-aware episodic retrieval differ from a recent window?
Role-aware Heuristic Episodic Attention for Conversational LLMs
Show answer and example
Answer: It can retrieve distant relevant interactions instead of using recency alone, and retains persistent instructions independently. Role policies treat content differently; compression may lose information.
Illustrative example: Recover an older preference while dropping recent irrelevant tool logs.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Basics
Why not top-k system instructions with chat history?
Role-aware Heuristic Episodic Attention for Conversational LLMs
Show answer and example
Answer: Global instructions persist across turns. Episodic content is retrieved separately and handled by role.
Illustrative example: A lasting formatting requirement should not be evicted for low similarity to the current query.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Core mechanism
Why can distilling from descriptions reduce disruption to the text backbone when adding media?
Omni-Embed-Mini: Binding Modalities Without Forgetting via Dense Distillation
Show answer and example
Answer: A frozen text backbone supplies stable semantic coordinates. Dense descriptions become teacher targets; a projector and controlled LoRA learn media mappings into that space. Stop gradients through the targets, and check for relevant media information that descriptions omit.
Illustrative example: An action-and-scene description anchors a video embedding, but missing small text cannot be recovered from the target alone.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Implementation details
Where should stop-gradient apply in cosine distillation?
Omni-Embed-Mini: Binding Modalities Without Forgetting via Dense Distillation
Show answer and example
Answer: Detach the description target from the frozen text backbone. Cosine loss updates only allowed media-side parameters. The local same-space dense-description diagnostic does not train the full six-modality encoder or reproduce retrieval leaderboards.
Illustrative example: Reduce the angle by moving the media mapping only. Updating both vectors would move the supposedly fixed semantic anchor.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Method comparison
What is the trade-off between frozen-text distillation and joint full-parameter multimodal training?
Omni-Embed-Mini: Binding Modalities Without Forgetting via Dense Distillation
Show answer and example
Answer: Freezing offers stable coordinates for new modalities and limits changes to existing text representations. Full training has broader flexibility but can shift old text capabilities. The scope of updates alone does not determine which performs better.
Illustrative example: Preserving existing text vectors makes an audio retrieval index easier to maintain, but text-space constraints can still limit audio-specific distinctions.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Basics
Does Omni-Embed-Mini output new media or a retrieval vector?
Omni-Embed-Mini: Binding Modalities Without Forgetting via Dense Distillation
Show answer and example
Answer: It learns a semantic media embedding comparable to description embeddings, not image, audio, or video generation. Dense descriptions supply training targets, but the media path must still encode the input itself.
Illustrative example: An audio vector comparable to a text query is representation learning; synthesizing sound from text is a generation task.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Core mechanism
Why re-estimate FFN channel importance after pruning attention paths?
MWOP: Modality-aware Width-wise Operation Pruning for Efficient MLLMs
Show answer and example
Answer: Pruning changes FFN input distributions, making old rankings stale. MWOP treats paths such as vision→vision and text→vision separately and prunes FFN channels by modality. Measure quality and actual speed independently.
Illustrative example: Removing text-to-vision paths can change how much fusion channels contribute.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Implementation details
Why not rank MWOP pruning by |w| rather than |w·∂L/∂w|?
MWOP: Modality-aware Width-wise Operation Pruning for Efficient MLLMs
Show answer and example
Answer: The product captures magnitude and current loss sensitivity. Magnitude alone omits sensitivity; absolute value prevents signed contributions from canceling. This local estimate depends on calibration data and path, not a guarantee deletion is harmless.
Illustrative example: Two equal weights with different gradients rank differently under the product but tie by magnitude.
Sections in the Chinese source: 核心公式, 复现边界. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Method comparison
What changes with modality-specific FFN pruning instead of one common channel mask?
MWOP: Modality-aware Width-wise Operation Pruning for Efficient MLLMs
Show answer and example
Answer: Text and vision tokens can retain different channel sets and capacities. One mask may hide different channel contributions. Sparse masking needs execution support; it does not guarantee hardware speedup.
Illustrative example: Text-related channels and texture channels may not overlap. Separate masks preserve both kinds, but the kernel must actually skip work.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Basics
Why can one MWOP attention head have different modality-path importance?
MWOP: Modality-aware Width-wise Operation Pruning for Efficient MLLMs
Show answer and example
Answer: One head can support vision→vision, text→vision, and text→text interactions with unequal contributions. MWOP assesses and prunes paths within a head rather than treating it as indivisible.
Illustrative example: Keep text reading vision in a head that contributes little to vision-only interaction.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Structured Decisions / Typed Readout: From Generated Sentences to Candidate Distributions
Core mechanism
Why normalize joint option-string probabilities rather than use the last token?
LLM2Jev: LLMs Are Already Jev-Style Decision Models -- When and How to Fine-Tune Them
Show answer and example
Answer: A multi-token option needs its full conditional chain. Fine-tuning adds a listwise objective and a base-model auxiliary KL anchor. Length and tokenization affect readout; the local candidate kernel does not evaluate a full checkpoint.
Illustrative example: Example: For a two-token marker use both conditional probabilities. Taking only the last can rank choices differently.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Structured Decisions / Typed Readout: From Generated Sentences to Candidate Distributions
Implementation details
How should LLM2Jev use the reference for auxiliary KL?
LLM2Jev: LLMs Are Already Jev-Style Decision Models -- When and How to Fine-Tune Them
Show answer and example
Answer: Freeze the base distribution and compute KL from base to auxiliary output. Updating the reference with the policy moves the anchor and changes the regularizer.
Illustrative example: Example: As the student changes, the base stays fixed instead of chasing the student.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Structured Decisions / Typed Readout: From Generated Sentences to Candidate Distributions
Method comparison
How does the LLM2Jev KL anchor differ from training only a listwise loss?
LLM2Jev: LLMs Are Already Jev-Style Decision Models -- When and How to Fine-Tune Them
Show answer and example
Answer: Listwise loss trains current choices; auxiliary KL against the base distribution limits drift on other outputs. It is a stability tradeoff, not a guarantee of retaining all prior abilities.
Illustrative example: Example: Tool selection may improve under the auxiliary constraint, but open-ended tasks still need independent evaluation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Structured Decisions / Typed Readout: From Generated Sentences to Candidate Distributions
Basics
How can LLM2Jev use an existing LLM for finite decisions?
LLM2Jev: LLMs Are Already Jev-Style Decision Models -- When and How to Fine-Tune Them
Show answer and example
Answer: Read probabilities of option markers to form a distribution without generating an explanation. Fine-tuning adds listwise supervision. This changes readout/optimization, not the need for inference.
Illustrative example: Example: Read numbered-marker probabilities to select a tool instead of generating its full name.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Core mechanism
Why retain projection bases for one IrekoGPT checkpoint to support nested widths?
IrekoGPT: Turning Structured Pruning into Post-Hoc Slimmable LLMs
Show answer and example
Answer: Dropping directions fixes width. Ordered bases let each width take a prefix, and multi-ratio calibration and linear correction reduce error. Shared structure does not equal shared quality or measured throughput.
Illustrative example: Switch between narrow and wide subnets only if extra basis directions were not discarded.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Implementation details
Why not fit an independent projection basis per width?
IrekoGPT: Turning Structured Pruning into Post-Hoc Slimmable LLMs
Show answer and example
Answer: W_k=P_:kᵀWP_:k requires prefixes of one basis. Independent choices break nesting. Preserve downstream linear calibration and ridge correction; local scope is subnet diagnosis.
Illustrative example: The 32-wide basis must be the first 32 directions of the 64-wide basis.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Method comparison
How do nested widths differ from single-ratio pruning in deployment?
IrekoGPT: Turning Structured Pruning into Post-Hoc Slimmable LLMs
Show answer and example
Answer: Single-ratio pruning retains one width. Nested storage and calibration allow runtime k selection, but do not guarantee every width's quality or speed.
Illustrative example: Validating k=64 does not validate k=32 automatically.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Basics
Why expose nested widths from one checkpoint?
IrekoGPT: Turning Structured Pruning into Post-Hoc Slimmable LLMs
Show answer and example
Answer: One ordered basis supports several budget levels without separately stored pruned models. Width k takes its first k directions and receives calibration.
Illustrative example: Narrow and wide paths share the same ordered basis.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Core mechanism
Why not apply ordinary Muon to tied embeddings?
AF-Muon: An AdamW-Free Muon Optimizer for Tied-Embedding Models
Show answer and example
Answer: Tied vocab weights support sparse input lookup and dense output classification with different gradient geometry. AF-Muon uses spectral updates for hidden matrices, support-aware capped directions for vocab, and RMS for vectors. Output gradients can touch rows absent from the input.
Illustrative example: An unseen input word can still receive classification gradient as an output candidate.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Implementation details
How avoid NaN on zero-gradient vocab rows?
AF-Muon: An AdamW-Free Muon Optimizer for Tied-Embedding Models
Show answer and example
Answer: Branch explicitly: zero gradients get zero directions. Multiplying by zero after computing g/‖g‖ cannot clear NaN. Local checks verify support and cap, not paper-scale pretraining.
Illustrative example: A row absent from both input and output gradients must not compute 0/0.
Sections in the Chinese source: 核心公式, 复现边界. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Method comparison
How does support-aware capped direction differ from unit-normalizing every vocab row?
AF-Muon: An AdamW-Free Muon Optimizer for Tied-Embedding Models
Show answer and example
Answer: It preserves zero support and caps row magnitude rather than forcing every nonzero row to equal length. Unit normalization can amplify small gradients and mishandle unsupported rows.
Illustrative example: Do not enlarge a small gradient merely to give it the same norm as a large row.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Basics
Why use different updates for different parameter types?
AF-Muon: An AdamW-Free Muon Optimizer for Tied-Embedding Models
Show answer and example
Answer: Hidden matrices, tied vocabulary, and one-dimensional parameters have different geometry and support. Use Muon, support-aware capped vocabulary directions, and RMS respectively.
Illustrative example: A sparsely visited input embedding row differs from a dense hidden matrix.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Core mechanism
Why add a consensus challenger after multiple verifiers agree?
VeriHarness: Scaling Agentic Verification for Long-Horizon Tasks
Show answer and example
Answer: Consensus can share mistakes or omissions. Resolving explicit disagreement cannot find requirements everyone skipped. The challenger questions consensus and gaps; the resolver gathers evidence for conflicts. Use evidence tools rather than majority counts as truth.
Illustrative example: If three reviewers inspect screenshots but none examines downloaded files, the challenger can flag missing artifact review even without disagreement.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Implementation details
Why record claim segmentation when implementing the VeriHarness claim-count penalty?
VeriHarness: Scaling Agentic Verification for Long-Horizon Tasks
Show answer and example
Answer: The score includes epsilon times claim count, so splitting or merging content changes the penalty. Preserve support, contradiction, and count to explain selection. Fixture verdicts are not real-world fact verification.
Illustrative example: Splitting one sentence into three claims changes the penalty; use consistent segmentation across candidates.
Sections in the Chinese source: 核心公式, 复现边界. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Method comparison
How does VeriHarness evidence scoring differ from simple majority voting?
VeriHarness: Scaling Agentic Verification for Long-Horizon Tasks
Show answer and example
Answer: Voting counts opinions. The score also considers supporting evidence, contradictions involving consensus, and a claim-count penalty. More votes or claims do not improve verifiability automatically. The local fixture checker is not the paper's actual tool environment.
Illustrative example: One precise, supported claim can be preferable to ten repeated but contradictory claims.
Sections in the Chinese source: 核心公式, 复现边界. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Basics
Why does VeriHarness challenge consensus as well as resolve disagreement?
VeriHarness: Scaling Agentic Verification for Long-Horizon Tasks
Show answer and example
Answer: Multiple matching answers can share missing evidence or omitted requirements. The resolver investigates disputed claims; the challenger questions shared claims and gaps. Evidence tools and workspaces keep agreement from substituting for correctness.
Illustrative example: Three agents can cite the same unchecked claim, which still needs verification despite unanimous votes.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Core mechanism
Why select the currently executable version separately from the next edit source?
Safety Must Survive Self-Improvement: Why Failures Persist and How Agents Recover
Show answer and example
Answer: A promising edit parent may not run safely under present conditions. Revalidate execution candidates and select from the qualified set. If none pass, return to founder rather than using a failed incumbent. Restoring execution and continuing search differ, and historical passes are not permanent eligibility.
Illustrative example: Keep a failed incumbent for edit analysis if useful, but execute a currently qualified version or roll back to founder.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Implementation details
Can Safe Self-Improvement reuse a candidate's previous pass flag directly?
Safety Must Survive Self-Improvement: Why Failures Persist and How Agents Recover
Show answer and example
Answer: No. Recheck current conditions and choose the highest-scoring current pass. The local path executes verification and founder fallback selection, without the paper's edit model or Bedrock experiments.
Illustrative example: Changed tool permissions or task conditions invalidate unconditional reuse of an old passed flag.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Method comparison
How does founder rollback differ from continuing with the incumbent when nothing passes?
Safety Must Survive Self-Improvement: Why Failures Persist and How Agents Recover
Show answer and example
Answer: Select the highest score only among currently valid candidates. If none qualify, roll back to founder. Continuing with a failed incumbent bypasses validation; rollback is a recovery protocol, not proof founder is always safe in any new condition.
Illustrative example: A top-scoring candidate that currently fails cannot keep the execution role on the strength of past scores.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Basics
Why does Safe Self-Improvement distinguish the next edit source from the current implementation?
Safety Must Survive Self-Improvement: Why Failures Persist and How Agents Recover
Show answer and example
Answer: A program can be a useful starting point without meeting current execution conditions. Detection, executable selection, and edit-source selection are separate, and candidates are revalidated under current conditions so old passes do not conceal current failure.
Illustrative example: Useful code from an old version does not make the entire version safe to run today.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Core mechanism
Why doesn't safe memory ingestion guarantee that its content has authority at execution time?
PACE: Provenance-Aware Capability Enforcement for Tool-Using LLM Agents
Show answer and example
Answer: Ingestion checks content then; execution also limits real effects under the authenticated request and verifies trusted provenance. Both authority and provenance must pass so untrusted memory cannot use a legitimate tool beyond scope. Text claiming authorization does not replace system authentication or per-effect checks.
Illustrative example: A stored webpage that says to delete files does not authorize deletion, even if ingestion found no danger. Check effects and provenance before execution.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Implementation details
Can PACE execute a tool first and apply the gate later from logs?
PACE: Provenance-Aware Capability Enforcement for Tool-Using LLM Agents
Show answer and example
Answer: No. A pre-effect gate must obtain the complete effect set before execution and compare authority and provenance. Later logs only audit; they cannot undo effects. The local path tests selection without operating system or browser tools.
Illustrative example: Finding untrusted provenance after an email is sent misses the opportunity to prevent it.
Sections in the Chinese source: 背景与主要改动, 核心公式, 复现边界. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Method comparison
Can PACE use capability or provenance checks alone?
PACE: Provenance-Aware Capability Enforcement for Tool-Using LLM Agents
Show answer and example
Answer: Allow an effect only if authority covers it and provenance is trusted. Capability limits what can be done; provenance checks whether the present basis is trustworthy. Both are required.
Illustrative example: General file-read permission does not imply approving this specific read instruction from an untrusted page.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Basics
Why does PACE check capability and provenance before tool side effects?
PACE: Provenance-Aware Capability Enforcement for Tool-Using LLM Agents
Show answer and example
Answer: Trusted stored material does not grant authority for every action. Derive authority from authenticated requests and check it with input provenance before each effect, preventing low-trust content from expanding permissions.
Illustrative example: A webpage saying 'send the file' is not user authorization to send it.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Core mechanism
Why reintroduce the original task and plan and check artifacts after the model says it is done?
Mingbird: A Local-First Agent Harness Enabling Small Open Models to Complete Real Tasks
Show answer and example
Answer: A long trajectory can forget requirements or confuse executed actions with delivery. Checking the original exposes missing files, missing verification, and early stopping. Loop detection and bounded format rescue address runtime faults, not model knowledge. Budget actual UTF-8 bytes rather than character counts.
Illustrative example: Example: Creating a file does not satisfy a task that also requires a test report. Re-read the original artifact list.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Implementation details
How are duplicate-call signatures and prefill budget computed?
Mingbird: A Local-First Agent Harness Enabling Small Open Models to Complete Real Tasks
Show answer and example
Answer: Use tool name and canonical JSON arguments, so key order does not distinguish calls. Block the sixth repeat before execution and correct after fifteen consecutive no-output actions. Prefill counts UTF-8 bytes. JSON rescue parses registered tools only without eval; this is not a full GUI product or sandbox.
Illustrative example: Example: Reversed key order cannot evade duplicate detection. A CJK character can use multiple UTF-8 bytes, so character counts understate input.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Method comparison
What do domain routing and flattened schemas trade off against showing all original tools?
Mingbird: A Local-First Agent Harness Enabling Small Open Models to Complete Real Tasks
Show answer and example
Answer: Routing narrows the tool domain. Presenting required arguments when schemas are large reduces formatting burden and irrelevant interfaces. Optional arguments can still matter, and user constraints cannot be dropped. This changes harness input and recovery, not base architecture.
Illustrative example: Example: A tool has ten parameters and two required ones. Start with a simplified schema but still honor a user-requested optional filter.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Basics
Why does MingBird focus on small models doing long multi-tool tasks?
Mingbird: A Local-First Agent Harness Enabling Small Open Models to Complete Real Tasks
Show answer and example
Answer: Small models have limited context, tool understanding, and error recovery. Loading all tools and all history adds pressure. MingBird organizes tools, execution loops, and completion checks under resource constraints. Components do not establish reliable arbitrary long-horizon capability; verify actual deliverables.
Illustrative example: Example: A file-and-database task first shows the relevant domain, then executes steps and verifies output rather than reading the whole catalog and declaring completion.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why filter by the question's time before reading instead of telling the model not to use future information afterward?
Mem++: Non-Destructive Memory for Long-Term Organizational LLM Agents
Show answer and example
Answer: Future documents can influence the model once they enter context, even with an instruction not to use them. Time eligibility first restricts lexical and semantic ranking to documents available then. Retaining verbatim text, date, and author aids audit but does not guarantee retrieval of the right record.
Illustrative example: Example: A March organizational decision cannot use an April announcement, however similar it is, as evidence of what was known in March.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
Did the local full-write and hybrid-retrieval test run OrgMemBench or an external embedding service?
Mem++: Non-Destructive Memory for Long-Term Organizational LLM Agents
Show answer and example
Answer: No. It runs verbatim retention, as-of filtering, and a reference lexical-semantic blend. Three seeds verify mechanism invariants. Organizational memory outcomes are the authors' benchmark; preset vectors and fixed records do not establish real assistant gains.
Illustrative example: Example: A future record being filtered verifies the date rule. Without real organizational QA and encoders, there is no comparable OrgMemBench capability score.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
Why keep both lexical and semantic ranking in Mem++?
Mem++: Non-Destructive Memory for Long-Term Organizational LLM Agents
Show answer and example
Answer: Lexical retrieval is good for exact names, IDs, and quotations. Semantic retrieval handles paraphrases and vocabulary changes. Fusion uses both on time-eligible original records and cannot create missing facts. Returned evidence may overlap or conflict, so the final answer must verify contents and provenance.
Illustrative example: Example: A ticket number can hit lexically. A query about the last database outage may match a record saying storage service interruption semantically.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Why do Mem++ non-destructive writes retain date and author?
Mem++: Non-Destructive Memory for Long-Term Organizational LLM Agents
Show answer and example
Answer: Original text carries facts, date supports as-of queries and version selection, and author preserves provenance and context. The write does not irreversibly rewrite records with a model, so later questions can reinterpret them. Raw storage does not guarantee retrieval and costs more storage and reads.
Illustrative example: Example: Keeping the dates and authors of two policy updates helps distinguish an older suggestion from a later official notice instead of blending them.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Core mechanism
What does giving newly available successor-state actions add beyond the next observation?
Code Owns the Simulation, Jev Owns the Evaluation
Show answer and example
Answer: The next observation describes the immediate change. Newly available actions show which future paths the step unlocks, exposing long-term value in actions without immediate payoff. Code simulates the transition on a cloned state, and the decision model evaluates public consequences only. Simulator reward or the reference plan must not leak into the scorer.
Illustrative example: Example: Picking up a key does not finish the task, but it enables opening a side door. Seeing that action reveals why the key can matter later.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Implementation details
Does returning a different object from fork guarantee branch isolation?
Code Owns the Simulation, Jev Owns the Evaluation
Show answer and example
Answer: No. Nested mutable state, random generators, and external resources must also be isolated. Distinct object identities do not justify sharing live files or simulator state. The step exposes only observable text, with no reward, gold plan, or success label. The selected action must be legal and is executed by the caller.
Illustrative example: Example: Two shallow copies share a backpack list, so picking up a key in the clone mutates the real backpack. Different objects did not isolate lookahead.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Method comparison
What does one-step simulation add over semantic action scoring, and why doesn't it guarantee optimality?
Code Owns the Simulation, Jev Owns the Evaluation
Show answer and example
Answer: Semantic scoring mainly uses action names and the task description. One-step simulation also gives the next observation and newly available actions, revealing unlocked opportunities. It still looks only one step ahead and the scorer may misjudge, so the new action list is not a long-horizon planning oracle.
Illustrative example: Example: Picking up a key may not seem close to the goal until simulation reveals the open-door action. Unknown obstacles behind the door can still prevent success.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Basics
What do code simulation and model evaluation each do in lookahead?
Code Owns the Simulation, Jev Owns the Evaluation
Show answer and example
Answer: Code executes legal actions on independent copies of the current state and produces actual successor observations. The model compares public consequences and selects an action. This avoids relying solely on textual guesses about transitions, but requires faithful isolation and simulation. Selection itself must not mutate the live environment.
Illustrative example: Example: Open a drawer on a copy to see its contents before deciding whether to open the real one. The scorer need not invent what is inside.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Core mechanism
Why does topology-aware role credit use workflow structure instead of averaging terminal reward across all roles?
It Takes Workflows to Evolve Better Workflows
Show answer and example
Answer: Roles occupy different positions and affect different downstream steps. Equal credit can conceal a bad node or a critical upstream role. Topological weights incorporate validity and successors, supporting both single-role changes and upstream/downstream collaboration in one framework. This is an attribution rule, not identification of causal contributions; use execution comparisons to test it.
Illustrative example: If a retriever's error reaches three downstream roles, repairing it may matter more than polishing the final paragraph. The topology shows the propagation range instead of dividing reward by headcount.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Implementation details
What should an implementation of valid_i(1+children_i) check, and what was not completed locally?
It Takes Workflows to Evolve Better Workflows
Show answer and example
Answer: Invalid roles get zero weight. Normalize over all valid weights, verify conservation of credit, and handle the no-valid-item case. The local path only executes this allocation operator; it does not generate workflows, train Qwen3.5, or run DataWright. Passing structural tests does not establish improvement from multi-agent co-evolution.
Illustrative example: An invalid node with ten children should receive no credit. When all nodes are invalid, handle the case explicitly rather than dividing by zero.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Method comparison
Which participants change in single-role evolution, upstream–downstream collaboration, and joint multi-role evolution?
It Takes Workflows to Evolve Better Workflows
Show answer and example
Answer: A single-role update fixes the other participants to isolate a local change. Upstream–downstream collaboration targets the dependency interface. Joint changes can improve coordination across roles, but their effects are harder to attribute. FlowRight expresses those scopes with a common workflow and credit representation. Success at one scope does not guarantee every joint update works.
Illustrative example: Changing just the writer and changing both retriever and writer may require different validation. The pair may adapt to a new evidence format, so gains cannot be attributed entirely to writing.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Basics
Why is it hard to decide which role to improve if a multi-agent workflow gets only a terminal result?
It Takes Workflows to Evolve Better Workflows
Show answer and example
Answer: Success depends jointly on roles and dependencies. Giving everyone the same reward hides who moved the workflow forward or held it back. FlowRight organizes role credit using workflow structure to support improvements at different scopes. Structural weights are an attribution rule, not direct measurements of each role's causal contribution.
Illustrative example: A retriever finds good evidence but a writer misreads it. Identical failure scores could lead subsequent updates to damage a useful retrieval habit as well.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Core mechanism
Why retain the propagation subgraph when locating a decisive error instead of labeling only the last failed action?
DeFA: Dependency-Guided Failure Attribution for LLM Agents
Show answer and example
Answer: Final failure may stem from earlier facts, parameters, or agent messages. Backward dependencies connect the source with affected nodes. Protocol relations give structure and semantic dependencies capture information flow. Maximum error scores still need evidence and do not prove complete causality.
Illustrative example: Deployment can fail because an early read used stale configuration. A propagation graph follows stale fields into the final arguments instead of blaming only deployment.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Implementation details
Why retain dependencies and responsibility rather than only event text in DeFA propagation graphs?
DeFA: Dependency-Guided Failure Attribution for LLM Agents
Show answer and example
Answer: Candidates come from Ancestor(V); missing edges hide upstream sources and wrong edges add unrelated events. Preserve protocol/semantic dependencies, event IDs, error magnitudes, and responsible agents. The local path reads no gold plan and runs no multimodal judge.
Illustrative example: If two agents write a file, trace the specific version consumed by the failure to identify the relevant write.
Sections in the Chinese source: 背景与主要改动, 核心公式, 复现边界. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Method comparison
How does DeFA choose a decisive error differently from taking the last failure?
DeFA: Dependency-Guided Failure Attribution for LLM Agents
Show answer and example
Answer: It selects by error magnitude among ancestors of the violation and retains responsibility and propagation. The last event is just a time position. Graph selection depends on supplied edges and error definition, not independent causal proof.
Illustrative example: A final retry timeout may be a symptom of wrong parameters generated earlier.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Basics
Why does DeFA trace backward along dependencies from the final violation?
DeFA: Dependency-Guided Failure Attribution for LLM Agents
Show answer and example
Answer: The last error may be downstream of a propagated mistake. Combine protocol and semantic relationships into an event graph, trace the source, and retain downstream propagation and responsible agent to distinguish trigger from decisive error.
Illustrative example: The deploying agent is not necessarily at fault if earlier configuration was written incorrectly.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why not delete a memory just because it often appears in failed tasks?
Causal Memory Policy: Making Memory Utility Identifiable by Intervening on Retrieval
Show answer and example
Answer: Hard tasks can preferentially retrieve that memory, biasing the correlation. Balanced randomized exposure gives comparable treatment and control observations, and standard errors protect uncertain items. Replacing one entry in fixed slots changes exposure of others, so the effect is specific to the design rather than universally additive.
Illustrative example: Example: A manual used in the hardest incidents can have low success because incidents are difficult. Randomized exposure helps test whether the manual itself hurts.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
Why do fixed k exposure slots prevent interpreting effects as independent additive coefficients?
Causal Memory Policy: Making Memory Utility Identifiable by Intervening on Retrieval
Show answer and example
Answer: Exposing one more memory changes the others' distribution inside fixed slots. The measured effect includes replacement under the randomized design. Each row has k entries, column exposures are roughly balanced, and each arm needs at least two observations. Recovering synthetic utilities is not a long-term cross-query policy or real QA accuracy.
Illustrative example: Example: In two slots, adding A displaces another candidate. The result measures the net effect of A replacing some content, not A's isolated contribution.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
How does protecting uncertain memories differ from deleting any with a negative estimated mean?
Causal Memory Policy: Making Memory Utility Identifiable by Intervening on Retrieval
Show answer and example
Answer: With high variance and few samples, a negative mean can conceal a positive effect. CMP combines standard errors with asymmetric deletion cost, refuses estimates with inadequate support, and returns noop under infinite uncertainty. One negative outcome is not permanent forgetting.
Illustrative example: Example: Two treatment runs that happen to fail are insufficient to prove harm. Wide uncertainty calls for retention or more evidence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Why measure a memory's effect on execution before cleaning up a library?
Causal Memory Policy: Making Memory Utility Identifiable by Intervening on Retrieval
Show answer and example
Answer: Memory takes context and may help, hinder, or be irrelevant. Deletion needs evidence of outcome utility. CMP makes utility estimable with retrieval-exposure experiments and retains uncertain items. It studies interventions under a fixed design, not frequency-based archiving.
Illustrative example: Example: Two old guides are both frequent. One helps recover from errors and the other repeats background. Outcome comparisons support keeping them differently.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why do explicit unresolved requirements detect stalls more easily than a history summary?
Beyond Memory: Harnessing Long-Horizon Agents with Explicit Belief States
Show answer and example
Answer: A summary describes work done. Unresolved requirements make progress comparable. If successive actions do not reduce them, the agent may be cycling and can recover around the specific gap. Action count is not progress. Facts and requirements must update from actual observations, not confidence.
Illustrative example: Example: Five searches still have not verified a date. The unresolved item exposes that activity did not move the goal forward.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
Does Δ|unresolved|≥0 prove a task is unsolvable?
Beyond Memory: Harnessing Long-Horizon Agents with Explicit Belief States
Show answer and example
Answer: No. It means that the window did not reduce outstanding requirements. Necessary preparation, missing evidence, or a bad updater can cause it. Trigger targeted recovery rather than declaring unsatisfiability. The local run tests belief updates and stall detection without external tools or learned extraction; rule activation is not long-horizon success.
Illustrative example: Example: Preparing identity verification may leave requirements unchanged but enable the next query. Stopping immediately would cut off a legal path.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
How does Belief Trapping differ from checking for repeated actions?
Beyond Memory: Harnessing Long-Horizon Agents with Explicit Belief States
Show answer and example
Answer: Action repetition checks surface behavior. Belief stalling checks whether unresolved requirements decrease. Different actions can circle the same bad assumption, and the same query may help after a state change. Diagnose progress from facts and requirements, not action strings alone.
Illustrative example: Example: Three search phrases fail to confirm payment despite different wording. Checking again after a transaction settles may actually make progress.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Why does explicit belief track both facts and unresolved requirements?
Beyond Memory: Harnessing Long-Horizon Agents with Explicit Belief States
Show answer and example
Answer: Facts describe the currently believed world; unresolved requirements describe what the task still lacks. History text alone can mask repeated work as progress. An explicit set supports stall detection, but entries need evidence before removal.
Illustrative example: Example: The user's address is known, while inventory is unconfirmed. Rereading the address does not resolve inventory.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why should a judge correct compression timing, summary content, and next action together?
AutoCompact: Learning When to Compact Context in Long-Horizon Coding Agents
Show answer and example
Answer: These decisions interact. Early compression can lose evidence, missing facts can induce bad actions, and later behavior can squander a correct summary. Executing the corrected tuple lets training observe joint consequences. Summary fluency alone cannot establish sufficiency for long-horizon coding.
Illustrative example: Example: A clear summary omitting a parameter constraint can cause the next edit to break an interface. Correct both memory and action.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
After a judge corrects the tuple, should the environment execute the original action or the corrected one?
AutoCompact: Learning When to Compact Context in Long-Horizon Coding Agents
Show answer and example
Answer: Execute the corrected decision, summary, and action together so training records match consequences. SFT plus outcome RL needs real execution feedback. The local run verifies the three-field in-place correction contract without training a coding agent or running SWE containers. Field checks are not code-repair results.
Illustrative example: Example: If the judge changes 'submit now' to 'run the failing test first,' executing the old submit action makes its reward invalid evidence for the corrected policy.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
What is the trade-off between learned compression timing and a fixed length threshold?
AutoCompact: Learning When to Compact Context in Long-Horizon Coding Agents
Show answer and example
Answer: A fixed threshold responds only to capacity and cannot distinguish moments with fragile commitments or unfinished work. A learned decision can use task state but requires reliable supervision and can err. Evaluate actual downstream coding results and cost alongside context reduction.
Illustrative example: Example: Compression just after a tool return and before recording its conclusion is risky. Waiting until state is clear can preserve what matters.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Why shouldn't a long-horizon coding agent wait until context is full before considering compression?
AutoCompact: Learning When to Compact Context in Long-Horizon Coding Agents
Show answer and example
Answer: Work includes pending edits, test feedback, and delivery requirements. Bad timing can discard precisely what the next step needs. AutoCompact learns when to compress and how to continue, aiming for task continuity under context pressure rather than shorter text alone.
Illustrative example: Example: Forced compression just after a failing test appears may drop the key error before its cause is recorded. Choose timing with task progress in mind.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Core mechanism
Why must failure modes treated as nonstationary bandit arms allow discovery of new scenarios and revisiting old weaknesses?
ActiveSaddler: Automated Curriculum Learning for Agent Harness Optimization
Show answer and example
Answer: A harness repair changes the payoff of previous failures. A fixed curriculum may keep testing solved problems while missing newly exposed faults. The controller allocates budget using value and exploration terms, periodically introducing unseen scenarios to support both discovery and verification. Counts or priorities are curriculum signals, not model capability metrics.
Illustrative example: After fixing timeouts, practice them less often but occasionally retest for regressions while exploring previously uncovered permission errors.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Implementation details
Does the local dynamic arm scheduler run AutoSaddler's real repair chain?
ActiveSaddler: Automated Curriculum Learning for Agent Harness Optimization
Show answer and example
Answer: No. It creates failure arms, explores and exploits, and updates priorities. It does not call AutoSaddler or run real GAIA2 or Terminal-Bench tasks. Observation counts and selection weights verify scheduling, not harness repair from synthetic feedback. Account for real rollout costs separately.
Illustrative example: If lowering an arm's synthetic reward reduces its selections, the controller responds to changes. It does not show that any actual tool failure was resolved.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Method comparison
Do fixed hard-task sampling and dynamically reprioritizing failures after repairs face the same stationary rewards?
ActiveSaddler: Automated Curriculum Learning for Agent Harness Optimization
Show answer and example
Answer: Fixed sampling generally follows preset difficulty or frequencies. Repairs change the value of prior failures: formerly frequent problems may no longer deserve repeated investment, while new ones appear. Dynamic scheduling should update with this nonstationary feedback. Historical failure rates are not permanent estimates of current reward, and revisits may yield no new information.
Illustrative example: After a parser fix, old hard examples consistently pass. Filling the curriculum with them wastes budget; newly observed authentication issues may deserve more attention.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Basics
With a limited repair budget, why does ActiveSaddler choose which failures to expose next?
ActiveSaddler: Automated Curriculum Learning for Agent Harness Optimization
Show answer and example
Answer: A harness cannot repair all scenarios equally deeply at once, and repeatedly testing familiar problems misses new weaknesses. ActiveSaddler treats the training curriculum as a dynamic decision based on existing failures and repair feedback. Selecting a difficult scenario only creates a learning opportunity; it does not establish a repair, and scheduling counts are not real capability gains.
Illustrative example: File-editing failures have been fixed repeatedly, while browser recovery issues just appeared. The curriculum must choose whether to keep verifying old behavior or budget exploration for the new fault.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why does GEAR use a shared orthogonal basis instead of freely updating each code vector?
Generative End-to-end Ad Retrieval at Douyin
Show answer and example
Answer: A shared basis constrains codebook geometry and stabilizes relationships as it changes. Hierarchical generation selects coarse-to-fine codes, then context reranks collision groups. Structure neither eliminates collisions nor replaces ranking supervision. Local checks cover quantization, state, and gradients, not full ad retrieval.
Illustrative example: Update codes within one coordinate system, but still distinguish two ads in the same code with current user context.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Why align row-vector conventions and check B's orthogonality for GEAR's zB and inverse mapping?
Generative End-to-end Ad Retrieval at Douyin
Show answer and example
Answer: Projection, nearest-code selection, and reconstruction must use consistent transposes; equal square shapes can hide mistakes. Verify orthogonality, residual quantization, and collision mapping separately. Diagnostics do not cover production generator quality.
Illustrative example: After multiplying a row by B, reconstruct using the corresponding inverse, and test the round trip on a known vector.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
How does GEAR's collision reranker allocate discrimination differently from indefinitely enlarging the codebook?
Generative End-to-end Ad Retrieval at Douyin
Show answer and example
Answer: Larger vocabularies distinguish more items at discrete addresses. Reranking keeps finite codes and resolves within-bucket candidates using user context, saving code capacity but adding bucket compute and recall limits. Local mechanism checks and orthogonality errors do not measure recommendation quality.
Illustrative example: Rerank two same-code cameras by user needs rather than assigning unique large-vocabulary codes; oversized buckets still cost compute.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Why do GEAR BasisVQ and BasisRQ represent codes in a shared orthogonal basis?
Generative End-to-end Ad Retrieval at Douyin
Show answer and example
Answer: A common basis coordinates updates; orthogonality constrains rotation so codewords do not drift independently. Hierarchical generation supplies addresses and context reranks collisions. Orthogonality does not guarantee semantic independence or collision-free coding; test quantization and ranking separately.
Illustrative example: Two similar items can still share a bucket despite a stable coordinate system.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Core mechanism
Why set successor distance to zero at success but directly zero progress at failure?
T2SPO: Trajectory-to-Step Policy Optimization for Agentic Reinforcement Learning
Show answer and example
Answer: Success reaches the goal; failure only stops execution, so treating it as distance zero creates false progress. Score before updating successful history to avoid leakage. Handle time truncation according to GRPO/PPO.
Illustrative example: A tool-exhaustion exit is not zero steps to success and must not reward the final failure.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Implementation details
Does T2SPO optimizer=ppo provide a full PPO trainer?
T2SPO: Trajectory-to-Step Policy Optimization for Agentic Reinforcement Learning
Show answer and example
Answer: No. It changes truncation semantics only, no critic/GAE. GRPO zeroes truncated credit; PPO uses predicted real successor. Fit standardization/SVD only on historical supports.
Illustrative example: Changing an option string does not create a value network.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Method comparison
How does distance-improvement credit complement terminal success?
T2SPO: Trajectory-to-Step Policy Optimization for Agentic Reinforcement Learning
Show answer and example
Answer: Distance differences guide intermediate steps; terminal reward still measures outcome. Mask/scale imperfect forecasts. TabPFN prediction errors don't establish task success.
Illustrative example: An action can move toward the goal without completing it; still verify terminal result.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Basics
Why use remaining steps in successful traces as T2SPO distance?
T2SPO: Trajectory-to-Step Policy Optimization for Agentic Reinforcement Learning
Show answer and example
Answer: Successful history offers state/remaining-step support examples for a frozen in-context predictor, giving denser progress than terminal success. Failure cannot be relabeled as close to success.
Illustrative example: A state three steps from completion provides a support label.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Core mechanism
Why use bounded scaling after rescoring segments against a successful path?
SHARPO: Segment-Level Credit Assignment for Agentic Reinforcement Learning
Show answer and example
Answer: Same-group success supplies extra context at the actual prefix. Segment logp gaps express local support. Bounded factors limit amplification and preserve original GRPO direction. Success context remains training-only.
Illustrative example: Retrieval in a failed trace may resemble a successful path while synthesis diverges, so weight the two segments differently.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Implementation details
What constraints apply to SHARPO segments and gold plans?
SHARPO: Segment-Level Credit Assignment for Agentic Reinforcement Learning
Show answer and example
Answer: Average teacher–student gaps within matching interaction segments and broadcast there, excluding tool observations. Local segments use no gold plan or 7B training; fixtures aren't capability evidence.
Illustrative example: Mixing a tool call with a different turn's answer gives the wrong segment mean.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Method comparison
How does bounded SHARPO multiplication differ from adding teacher advantage?
SHARPO: Segment-Level Credit Assignment for Agentic Reinforcement Learning
Show answer and example
Answer: Twice sigmoid lies between zero and two, so scaling preserves sign and cannot turn zero nonzero. Additive terms can flip credit. The teacher modulates intensity rather than rewriting outcomes.
Illustrative example: A failed segment can be penalized less but not rewarded merely by teacher preference.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Basics
Why give SHARPO a successful same-group trajectory as teacher context?
SHARPO: Segment-Level Credit Assignment for Agentic Reinforcement Learning
Show answer and example
Answer: An in-task success guides rescoring student logp within interaction segments and scales original GRPO advantage. Teacher context is training credit, not a deployment gold plan.
Illustrative example: Use a successful path to strengthen related behavior without pasting it into the inference prompt.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Core mechanism
Why train the HAWK drafter on shifted trajectories after its own proposals?
HAWK: Rethinking Multimodal Drafting for Speculative Decoding
Show answer and example
Answer: Proposed tokens put the system on drafter-generated prefixes. Training only on teacher prefixes hides that shift. Shifted trajectories align supervision with actual proposal states, and a mix of teacher layers conveys visual information. The target must still verify acceptance.
Illustrative example: If the first draft token chooses different wording, the second should learn the target distribution under that actual prefix.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Implementation details
Does local HAWK validation include a specialized decoding kernel?
HAWK: Rethinking Multimodal Drafting for Speculative Decoding
Show answer and example
Answer: No. It checks softmax-weighted target hidden states, shifted-target supervision, and target isolation. It does not run a complete speculative verification loop or specialized kernel, so cannot demonstrate end-to-end lossless speedup.
Illustrative example: Checking that three weights sum to one does not measure accepted tokens or saved generation time.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Method comparison
How does HAWK's learned layer mix differ from using only the final target layer?
HAWK: Rethinking Multimodal Drafting for Speculative Decoding
Show answer and example
Answer: Softmax-normalized trainable weights combine hidden states from multiple layers; final-only is a fixed choice of one source. Weights sum to one but need not be equal. Mixing alone proves neither acceptance-rate gains nor speed.
Illustrative example: A high layer weight increases its share in the mixed representation rather than concatenating layers as extra tokens.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Basics
What are the target and drafter roles in HAWK speculative decoding?
HAWK: Rethinking Multimodal Drafting for Speculative Decoding
Show answer and example
Answer: The cheaper drafter proposes tokens; the target verifies them and preserves the output distribution. HAWK compresses multiple target layers for a shallow drafter to improve proposals, not to bypass verification.
Illustrative example: A shallow model proposes several words, then the larger model checks how many can be accepted.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Core mechanism
Why can't a mutator write back a coding rule based only on its own assessment?
Self-Evolving Coding Rules for AI Coding Agents
Show answer and example
Answer: A plausible rule may fit only a few examples or damage old behavior. An isolated validation evaluator measures the candidate pool and variants consistently and selects what to write back, separating proposals from acceptance evidence. Selection data are development feedback; do not expose final tests to repeated mutator tuning.
Illustrative example: A new rule that retries every function sounds robust but can repeat non-idempotent operations. Independent validation can reveal this regression.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Implementation details
Does local selection after mutation establish that LLM rule evolution ran?
Self-Evolving Coding Rules for AI Coding Agents
Show answer and example
Answer: No. It checks validation selection among mutated candidates but does not call an LLM mutator or full coding agent. Correct argmax over supplied scores is control-flow evidence; if scores come from fixtures, they do not indicate code quality. Retain old rules and failed candidates for audit.
Illustrative example: Choosing the largest of three preset scores verifies the selector. Without a model proposing rules and actually fixing code, it does not verify the whole self-evolution path.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Method comparison
How does retaining a rule candidate pool differ from immediately overwriting the only rule, for search and rollback?
Self-Evolving Coding Rules for AI Coding Agents
Show answer and example
Answer: A pool compares multiple versions under one evaluation protocol and retains old rules as baselines or candidates. Immediate overwrite makes the proposal active and can lose a usable baseline after failure. A pool does not supply a correct judge by itself; it still needs an isolated evaluator. Candidate counts are not actual performance on tool tasks.
Illustrative example: Keep both a retrieval-focused rule and a testing-focused rule for comparison. If each generation immediately overwrites the rule, it is hard to tell which change caused a regression.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Basics
Why does RuleEvolve treat coding rules as development experience that can be tested and revised?
Self-Evolving Coding Rules for AI Coding Agents
Show answer and example
Answer: Coding-agent rules affect recurring search, editing, and testing behavior, and one manually written tip might not fit the task distribution. RuleEvolve selects variants with behavioral evidence so experience can be revised. More detailed wording is not better execution; use isolated validation rather than the generator's confidence to update rules.
Illustrative example: After adding 'check callers before editing,' test whether it helps repair the same development tasks, not just whether it sounds sensible.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why store append-only raw turns separately from searchable segment summaries?
MemFit: Efficient Long-Term Agentic Memory
Show answer and example
Answer: Summaries make compact indexes but can omit details. Keeping raw records unchanged gives later close reading traceable evidence. Writes avoid expensive model rewrites of all history; reads combine lexical, semantic, and reranking signals. An index hit is only an entry point, and an omitted summary detail may still exist.
Illustrative example: Example: A segment summary says payment API discussion. Read the original turn to find the exact timeout; keeping only the summary might lose it permanently.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
Does the local hybrid retrieval hook mean embedding and cross-encoder inference ran?
MemFit: Efficient Long-Term Agentic Memory
Show answer and example
Answer: No. The hook accepts lexical, semantic, and reranking signals but does not call a real embedder or cross-encoder. It tests append-only behavior and score combination. Fixture semantic scores cannot establish real long-conversation retrieval quality or paper latency.
Illustrative example: Example: Correctly sorting three records with preset semantic scores tests fusion, not whether a model derives reliable scores from natural language.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
What does inexpensive append-only writing trade off against LLM memory rewriting every turn?
MemFit: Efficient Long-Term Agentic Memory
Show answer and example
Answer: Append-only avoids repeated generation and irreversible rewriting, but raw storage grows and retrieval and indexing do more filtering. LLM writes can abstract early but lose details. Compare writing, storage, and final retrieval quality together, not just call counts in one stage.
Illustrative example: Example: Saving a thousand raw turns is cheap, but a later query must quickly locate one address change. Inexpensive writes do not guarantee cheap reads.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Why does MemFit focus on write costs for high-frequency conversations?
MemFit: Efficient Long-Term Agentic Memory
Show answer and example
Answer: A long-lived assistant continually receives turns. Rewriting all memory with a costly model on every turn incurs cost before retrieval and can erase details. MemFit keeps writes lightweight and originals traceable, moving query-specific filtering to reads while facing the growing index size.
Illustrative example: Example: A support system gains hundreds of turns per day, many never queried. Complex semantic rewriting of each turn can cost more than useful future queries.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Core mechanism
Why keep several action branches rather than immediately selecting the highest score from the first distribution?
JevSpawn: Adaptive Agentic Inference through Compositional Action Spaces
Show answer and example
Answer: Early probabilities can be wrong. Keeping multiple composite branches leaves room to recover after contradictory evidence. New feedback reweights priors and guides future exploration; it does not rewrite past success labels. Branches consume budget, so parallel retention trades compute for lower decision risk rather than searching all possibilities.
Illustrative example: Example: Keep both searching and inspecting local files. If search fails, switch to the retained inspection branch without rebuilding the plan from scratch.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Implementation details
Why do p(a)·exp(feedback) updates still need normalization and a branch budget?
JevSpawn: Adaptive Agentic Inference through Compositional Action Spaces
Show answer and example
Answer: Multiplication yields unnormalized weights. Divide by their total to obtain a distribution, then retain branches within budget. Multiplication cannot recover a branch with zero prior mass, and extreme feedback can cause numerical instability. The local test verifies a finite posterior and top-branch retention without calling a Jev checkpoint.
Illustrative example: Example: Two branches begin at one half with feedback 0 and 1. Their weight ratio becomes 1:e; e/2 is not a valid probability.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Method comparison
How does branch retention differ from independently resampling plans every round?
JevSpawn: Adaptive Agentic Inference through Compositional Action Spaces
Show answer and example
Answer: Retention preserves the probabilities and states of earlier unchosen options for reuse after feedback. Independent resampling can repeat old errors or lose progress. Retention also consumes compute and memory, so more retained candidates do not prove lower cost; compare at equal budgets.
Illustrative example: Example: After search fails, switch to the existing file-inspection branch. Resampling may suggest the same failed search again.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Basics
What options does JevSpawn preserve when one agent execution path fails?
JevSpawn: Adaptive Agentic Inference through Compositional Action Spaces
Show answer and example
Answer: Early judgments in long tasks can be wrong. Spending the entire budget on one action makes contradictory evidence costly. JevSpawn allocates reasoning adaptively by organizing optional actions into retainable composite branches and shifting effort after execution feedback. This preserves alternatives without promising exhaustive search.
Illustrative example: Example: Inspecting local data and searching online can both be reasonable. If the first path is blocked, continue the other prepared path rather than starting from a blank plan.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why share recursive blocks in RECAP-CTR yet average routes and retain a training-trajectory EMA?
Beyond Interaction Capacity: Estimator Scaling with Recursive Models for CTR Prediction
Show answer and example
Answer: Different routes through shared blocks produce multiple logits without independent models. Averaging reduces route fluctuations; EMA smooths weights over training time. Correlated routes limit benefit. Local evidence verifies state and operator invariants only.
Illustrative example: Illustration: Two recursive routes yield slightly different click scores; average those while separately smoothing neighboring training checkpoints.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why checkpoint RECAP-CTR EMA state separately?
Beyond Interaction Capacity: Estimator Scaling with Recursive Models for CTR Prediction
Show answer and example
Answer: EMA accumulates parameter history across steps, so losing it changes the estimator on resume. Match route count and averaging denominator rather than treating missing routes as zero. Verify recurrence, EMA updates, and export separately.
Illustrative example: Illustration: Resetting EMA to current parameters after interruption loses thousands of smoothed steps even if training resumes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
Why can't route averaging replace trajectory EMA in RECAP-CTR?
Beyond Interaction Capacity: Estimator Scaling with Recursive Models for CTR Prediction
Show answer and example
Answer: Route averaging combines simultaneous outputs; EMA smooths parameters over time. Shared routes can remain correlated and EMA can lag drift. Local invariants do not show CTR lift from smoothing.
Illustrative example: Illustration: All routes can share a high bias, and EMA can smooth an unusual step without adding a missing feature.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Is averaging RECAP-CTR route logits equivalent to averaging click probabilities?
Beyond Interaction Capacity: Estimator Scaling with Recursive Models for CTR Prediction
Show answer and example
Answer: No. Sigmoid is nonlinear. The source averages logits first, which does not turn shared routes into independent complete models.
Illustrative example: Illustration: With logits 0 and 4, sigmoid of their mean is about 0.88; mean sigmoid is about 0.74.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why does HELIX forbid candidate tokens from writing back to shared sequence state?
Show answer and example
Answer: One-way dependence keeps expensive user-history encoding candidate-independent and reusable within a request while candidates repeatedly read it for deep interaction. Writes back would make history candidate-dependent. Local checks cover sensitivity and cache invariance, not production throughput or gains.
Illustrative example: A shoe can reuse the history read by a camera only if the camera did not modify that state.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why check candidate deltas, invariant user caches, and finite gradients in HELIX?
Show answer and example
Answer: Candidate changes should alter scores without changing public user state. Missing either indicates ignored candidates or cache pollution; finite loss and gradients test trainability. These are diagnostics, not quality metrics.
Illustrative example: Identical scores suggest candidate insensitivity; changing scores and cache together breaks sharing despite successful forward execution.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How do HELIX one-way communication and WHALE-style bidirectional exchange differ in cacheability?
Show answer and example
Answer: HELIX blocks candidate-conditioned writes into public sequence state. Bidirectional exchange allows deeper co-evolution but creates candidate dependence. Both can use cross-attention without equivalent dependencies, and no universal quality ordering follows.
Illustrative example: After a camera reads shared history, shoes can read the same cache only if the camera state never wrote back.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why does HELIX expand both sequence modeling and heterogeneous feature interaction?
Show answer and example
Answer: History and feature combinations supply distinct information. Interleaved modules let candidate states read history at multiple depths while preserving user-only state. This is not late concatenation, and caching constrains direction.
Illustrative example: Layers can combine travel history with price and device features; enlarging either path alone cannot provide the other's evidence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why activate GRP's one-sided logged-target margin only when original-target retrieval suffers?
Show answer and example
Answer: Reward optimization can move mass to a few high-reward outputs and lower logged-target probability. Penalize decreases past tolerance while permitting other improvements. Freeze and stop gradients through the ranking reward head to prevent co-adaptation. Local objectives and gradients do not verify production funnel replacement.
Illustrative example: Improve new candidates without a penalty if the original target remains competitive; penalize shifting all mass to one high-reward category.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Why does GRP's MHP ranking head read detached generator representations?
Show answer and example
Answer: Ranking gradients through shared encoding and SID paths can damage existing retrieval. Stop-gradient trains the ranker without changing the generator from this loss. Version it when frozen as RL reward and verify actual gradient separation rather than only registration.
Illustrative example: Updating ranking BCE should change the head but not send gradients into the SID-generation backbone.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
What freedom does GRP's one-sided margin retain compared with symmetric probability constraints?
Show answer and example
Answer: Only a logged-target drop beyond reference tolerance incurs penalty, leaving other preference improvements free. Symmetric constraints may suppress beneficial changes too. Target preservation does not validate the new reward, and local checks only cover surrogate objectives and gradients.
Illustrative example: Raise better candidates while preserving the logged target without penalty; a large abandonment of that target activates the guard.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Why migrate GRP's recommendation funnel gradually instead of replacing all stages at once?
Show answer and example
Answer: Established systems serve several goals and risks. Introduce an additional retrieval source, replace weak sources, increase quota, and then bypass ranking progressively. Keep original recall protections and separate evidence and permissions per stage; a unified design is not a finished rollout.
Illustrative example: Start with a small candidate share and expand only after validation rather than removing retrieval and fine ranking on one offline win.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why does CohortMix-TS use a mixed Beta prior with fixed strength instead of inheriting all cohort counts?
Show answer and example
Answer: Similar cohorts can warm-start a new user, but copying their sample sizes would make personal observations ineffective for too long. Mix cohort means, use controlled pseudocounts, and update on individual exposure feedback. Thompson sampling retains exploration, and local evidence only checks the posterior mechanism.
Illustrative example: With 100,000 cohort events and three personal events, full inheritance drowns out the new user. Fixed prior strength allows personal preference to take over gradually.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Why keep independent Beta counts per new user and exclude previously exposed slate items in CohortMix-TS?
Show answer and example
Answer: The cohort prior only initializes state. Subsequent success and failure belong to that user's arm, not every new user. Select from unseen items each round for sequential exploration; unexposed arms do not receive invented failures.
Illustrative example: A comedy click updates only the viewing user's success count. An action movie not shown receives no feedback.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
How does CohortMix-TS explore differently from always choosing the highest posterior mean?
Show answer and example
Answer: Greedy selection ignores uncertainty. Thompson sampling draws from each posterior, giving uncertain but promising arms a chance; more evidence naturally reduces uncertainty. It does not improve every round, and the local three-seed state-transition diagnostic is not recommendation capability evidence.
Illustrative example: A slightly lower-mean, rarely observed arm can sometimes beat a well-known arm in sampled scores, while greedy selection may never try it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why does CohortMix-TS construct a mixed cohort prior before sufficient personal data arrive?
Show answer and example
Answer: Few personal observations give unstable success estimates. Infer cohort memberships from metadata and mix historical item-arm means to guide initial exploration. Fixed strength limits borrowed confidence so individual feedback can gradually revise it. Shared membership does not imply identical preferences.
Illustrative example: Initialize a new user's movie success rates from two cohorts, then let a few personal outcomes move the posterior away from the group average.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Core mechanism
Why does an early violation persist instead of being diluted by easy later tokens?
The Weakest Link: Distilling LLM Reasoning with Worst-Case Constrained Reinforcement Learning
Show answer and example
Answer: Every prefix must meet teacher cost. Absorption blocks post hoc dilution; immediate counterfactuals vary only action at the real visited prefix. Teacher improbability still doesn't prove factual error.
Illustrative example: Adding common words after an unsupported derivation may lower final cost but cannot satisfy already violated prefixes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Implementation details
May counterfactual current actions use different historical costs?
The Weakest Link: Distilling LLM Reasoning with Worst-Case Constrained Reinforcement Learning
Show answer and example
Answer: No. Start from actual prefix cost and vary only current action. Exact full-vocabulary immediate gradients coexist with sampled future returns. No full LM trainer.
Illustrative example: Inventing a worse history for an unfavored action invalidates its comparison.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Method comparison
How does absorbing penalty differ from a recoverable moving gate?
The Weakest Link: Distilling LLM Reasoning with Worst-Case Constrained Reinforcement Learning
Show answer and example
Answer: Violation persists even if later average improves. A rolling gate might reopen. This is a prefix constraint, not proof teacher cost equals task value.
Illustrative example: One high-probability next token doesn't clear the previous violation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Basics
Why require every prefix to satisfy average cost in Weakest Link?
The Weakest Link: Distilling LLM Reasoning with Worst-Case Constrained Reinforcement Learning
Show answer and example
Answer: A final mean can hide bad early reasoning diluted by easy later tokens. First violation is absorbing. Costs are teacher negative logp.
Illustrative example: Appending common words cannot undo an unsupported prefix.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Core mechanism
Why use two SIPO teacher views rather than one correct hint?
SIPO: Unifying Reinforcement Learning with On-Policy Self-Distillation
Show answer and example
Answer: Contrast log probabilities on identical student rollouts to reduce shared confidence and style. Environmental reward supplies main direction and dense evidence redistributes token credit. Both views are training-only.
Illustrative example: Both teachers may prefer politeness but diverge on key evidence; contrast downweights the shared style.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Implementation details
Can SIPO have token signal when all group rewards are equal?
SIPO: Unifying Reinforcement Learning with On-Policy Self-Distillation
Show answer and example
Answer: Yes. Centered contrast evidence can be nonzero with zero outcome advantage. Gold stays in teacher/evaluator only, not rollout.
Illustrative example: A failed group can receive directional teacher evidence without becoming autonomous successes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Method comparison
What assumption differs between SIPO contrast and one privileged teacher?
SIPO: Unifying Reinforcement Learning with On-Policy Self-Distillation
Show answer and example
Answer: Single-teacher confidence can include irrelevant effects. Contrast aims to isolate conditioning differences, but poorly designed contexts still mislead and are not independent truth.
Illustrative example: Swapping irrelevant wording needn't produce a useful solving contrast.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Basics
Why provide positive and negative privileged contexts for each SIPO rollout?
SIPO: Unifying Reinforcement Learning with On-Policy Self-Distillation
Show answer and example
Answer: Teacher logp differences aim to cancel common confidence/style and expose informative token evidence. Environment reward still guides the task. Student sampling sees neither privilege.
Illustrative example: If both teachers favor a filler phrase, subtraction reduces that common bias.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Core mechanism
Why isn't hidden-residual extrapolation equivalent to increasing output probability ratios?
Show answer and example
Answer: The target is per-layer/token base plus scaled RL-teacher shift, an internal path probabilities do not preserve. Freeze teacher/base. α above one does not ensure capability beyond teacher.
Illustrative example: Extend RL's layer displacement rather than amplify all teacher-preferred output words.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Implementation details
Why stop gradients through RIDE teacher and base?
Show answer and example
Answer: The target is a fixed RL change; updating its branches makes residual drift. Align layers, tokens and dimensions before subtracting.
Illustrative example: Student error must not move the teacher target rather than update the student.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Method comparison
Does representation extrapolation with α>1 guarantee outperforming the teacher?
Show answer and example
Answer: No. Extending the direction may magnify benefits or move outside reliable states. Evaluate independently; farther geometry is not higher ability.
Illustrative example: Overshooting can destroy information despite extending the same direction.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Basics
Does RIDE extrapolate output probabilities or hidden residuals?
Show answer and example
Answer: It compares RL teacher with pre-RL base per layer/token, then regresses the student to base plus amplified residual. This is representational change, not logit ratios.
Illustrative example: Extend a learned layer direction rather than multiply favored word probability.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
Why debias and binarize a frozen critic instead of maximizing its continuous score?
Unlocking the Critic: Reward-Free Policy Optimization for LLM Post-Training
Show answer and example
Answer: Freezing stops parameter drift, not bias. Policies can exploit length or other shortcuts. Debias and binarize prefix success estimates while using the critic for baselines and unfinished forecasts. Reward-free reuses supervision rather than removes it.
Illustrative example: A length-biased critic is exploitable even when frozen.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
Why freeze the RFPO critic and distinguish unfinished forecasts?
Unlocking the Critic: Reward-Free Policy Optimization for LLM Post-Training
Show answer and example
Answer: Freezing keeps the evaluation baseline stable. A forecast handles missing terminal outcomes, not verified success. Local debiasing, binary reward and GAE kernels do not train a long-reasoning model.
Illustrative example: A high estimate on a truncated trace remains an estimate, not an answer labeled correct.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
What risk comes from sharing a critic for reward and baseline?
Unlocking the Critic: Reward-Free Policy Optimization for LLM Post-Training
Show answer and example
Answer: Shared bias can be exploited by the policy. Length debiasing and binarization limit shortcuts without ensuring attack resistance. Critic score itself isn't evidence of capability gains.
Illustrative example: If the critic favors verbosity, the policy may simply lengthen responses.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
Does RFPO reward-free mean no quality signal?
Unlocking the Critic: Reward-Free Policy Optimization for LLM Post-Training
Show answer and example
Answer: A calibrated frozen critic predicts prefix success instead of costly external terminal rewards. Supervision moves into the critic, whose reliability still matters.
Illustrative example: An unfinished solution can get a success estimate because of prior evaluation learning, not by knowing the answer from nothing.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Core mechanism
Why forward the same response with/without hindsight skill rather than imitate skill text?
PR-OPD: Privileged Representation On-policy Self-Distillation for Agentic Reinforcement Learning
Show answer and example
Answer: Compare how skill changes same-token/layer representations and transfer to an unprivileged student. Copying skill or teacher replies changes targets. Detach teacher, combine task reward and remove privilege at inference.
Illustrative example: A teacher can rescore an earlier decision using a validation reminder that the deployed student never sees.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Implementation details
What masks apply to layerwise PR-OPD alignment?
PR-OPD: Privileged Representation On-policy Self-Distillation for Agentic Reinforcement Learning
Show answer and example
Answer: Align valid tokens of the same response at corresponding layers, excluding observations/padding. Skills come from completed training traces, not gold plans. Local kernel isn't agent training.
Illustrative example: Do not align extra teacher skill tokens to student response positions.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Method comparison
How do two forwards differ from concatenating skill to student context?
PR-OPD: Privileged Representation On-policy Self-Distillation for Agentic Reinforcement Learning
Show answer and example
Answer: Run ordinary student and privileged detached teacher separately. Giving student skill changes deployment assumptions and can teach dependency. Representation matching isn't answer supervision.
Illustrative example: A tested student should rely on learned parameters when no skill text is supplied.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Basics
Why distill representations in PR-OPD rather than only final probabilities?
PR-OPD: Privileged Representation On-policy Self-Distillation for Agentic Reinforcement Learning
Show answer and example
Answer: Hindsight skills create privileged hidden states. Align projected states at matching layers and response positions, then combine with task reward. Teacher skill is removed at inference.
Illustrative example: A retrospective tool principle can guide internal features without being deployment input.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Core mechanism
Why can an unverified attempt be teacher context but not scaffold?
Overcoming Scaling Limits in On-Policy Self-Distillation for LLM Reasoning
Show answer and example
Answer: A scaffold fixes a credible student trajectory; teacher context only provides comparison clues. Unverified attempts aren't correct labels. No verified rollout means no training signal, and final verification doesn't reveal gold to students.
Illustrative example: Score prefixes of a concise verified response while allowing the teacher to reference a longer exploration.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Implementation details
Should OASIS order by length before or after checking eligibility?
Overcoming Scaling Limits in On-Policy Self-Distillation for LLM Reasoning
Show answer and example
Answer: Verify first, then choose the shortest eligible trajectory. Track scaffold and teacher context separately and hide future answers from student.
Illustrative example: A one-line wrong result must not beat a longer verified solution.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Method comparison
Why require a verifier if OASIS has no gold solution?
Overcoming Scaling Limits in On-Policy Self-Distillation for LLM Reasoning
Show answer and example
Answer: It needs a verified successful rollout before selecting a scaffold. Absence of gold steps does not remove correctness checks. With none verified, emit no signal.
Illustrative example: The shortest unchecked answer is not automatically a valid scaffold.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Basics
Why separate OASIS scaffold and teacher context?
Overcoming Scaling Limits in On-Policy Self-Distillation for LLM Reasoning
Show answer and example
Answer: Use the shortest verified trajectory for supervision locations, preferably another unverified attempt for teacher context. Where to learn and what extra context to use are independent.
Illustrative example: A concise success can anchor positions while a longer attempt provides alternative clues.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Core mechanism
Why use both top-k coverage and Bhattacharyya similarity for interventions?
From Dissonance to Orchestration: Teacher Intervention in On-Policy Distillation
Show answer and example
Answer: One checks support inclusion, the other shared probability allocation. Disagreement and position weight prioritize scarce teacher compute. It is a clue, not correctness; local PDS doesn't generate a takeover.
Illustrative example: Two models can share identical top-k words while assigning very different probabilities.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Implementation details
Does local MAESTRO generate takeover rollouts?
From Dissonance to Orchestration: Teacher Intervention in On-Policy Distillation
Show answer and example
Answer: No. It computes top-k coverage, normalized similarity and PDS. Real intervention needs generation, state handoff and cost accounting; no measured call savings.
Illustrative example: Hand-entered probabilities can trigger a score without any teacher rollout.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Method comparison
Why combine coverage and Bhattacharyya similarity?
From Dissonance to Orchestration: Teacher Intervention in On-Policy Distillation
Show answer and example
Answer: Coverage checks whether student candidates include teacher support. Similarity compares mass within shared support. Either alone misses differences; high PDS isn't teacher correctness.
Illustrative example: Identical candidates can have almost opposite weights.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Basics
Why intervene only at MAESTRO's high-disagreement prefixes?
From Dissonance to Orchestration: Teacher Intervention in On-Policy Distillation
Show answer and example
Answer: Teacher rollouts are expensive. Top-k coverage, similarity and early-position weighting allocate limited guidance. Disagreement doesn't automatically mean student error.
Illustrative example: An early fork can matter more than a punctuation discrepancy.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Core mechanism
Why combine a language prior and visual likelihood instead of averaging teachers?
LEGO-OPD: Factorized Teacher Composition for Multimodal On-Policy Distillation
Show answer and example
Answer: Language supplies general grammar/reasoning while vision modifies evidence through a product. This can avoid copying visual model language bias if its likelihood is reliable. Local target checks do not prove vision.
Illustrative example: Language makes a cup description grammatical; visual evidence increases the color it shows.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Implementation details
Does local LEGO load both language and visual experts?
LEGO-OPD: Factorized Teacher Composition for Multimodal On-Policy Distillation
Show answer and example
Answer: No. It only combines factors. Align candidates, add in log space, then normalize. Visual likelihood isn't a full VLM posterior; random logits aren't visual capability.
Illustrative example: Multiplying two vectors doesn't make any model see an image.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Method comparison
How does setting grounding α=0 differ from using only vision?
LEGO-OPD: Factorized Teacher Composition for Multimodal On-Policy Distillation
Show answer and example
Answer: A valid zero-weight visual contribution leaves language prior. Vision-only would remove language, a different target. α scales evidence, not votes between two complete models.
Illustrative example: Multiplying visual log contribution by zero leaves language scores, not pure vision.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Basics
Can LEGO-OPD grounding likelihood be read as the final answer probability?
LEGO-OPD: Factorized Teacher Composition for Multimodal On-Policy Distillation
Show answer and example
Answer: No. It is the visual evidence factor, combined with language prior and normalized. Distinguish prior, likelihood and final posterior, not interpret one score as answer confidence.
Illustrative example: Vision supports red; language constrains valid next words. Combine and normalize for token probabilities.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Core mechanism
Why isn't mixing each token probability the same as selecting one model for the whole reply?
Show answer and example
Answer: Interpolate at every updated prefix; sampled tokens change later states. Choosing a model once gives a different sequence law, and averaging logits is also different. Endpoint checks do not validate the paper's speculative sampler.
Illustrative example: Sample a teacher-favored token first, then remix under the new prefix rather than fix a source for the rest.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Implementation details
Do local IPD endpoint checks include speculative high-throughput sampling?
Show answer and example
Answer: No. Exact token mixture checks endpoints, normalization and total variation. γ=0 student, γ=1 teacher. Verification-aware speculative sampler and full training remain unreproduced.
Illustrative example: Calculating a mixture vector doesn't establish paper sampling throughput.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Method comparison
Is token-level mixing equivalent to choosing one model per rollout?
Show answer and example
Answer: No. Tokenwise mixtures can combine both preferences within a sequence; one source per rollout produces a different joint state distribution.
Illustrative example: Student-like first step and teacher-like second may be reached differently under fixed rollout source.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Basics
Why interpolate IPD teacher/student sampling distributions?
Show answer and example
Answer: Pure student and teacher states trade coverage differently. Mix conditional policies continuously at each state; γ controls sample source distribution, not merely loss weighting.
Illustrative example: Let some steps follow student tendencies and other steps gain teacher-supported options.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
Why exchange full same-prompt peer groups instead of only a winning answer?
Learning Beyond What You Sample: Off-Policy-Aware Cross-Model Trajectory Exchange for RLVR
Show answer and example
Answer: A receiver all-fail group lacks differentiation; the whole peer group preserves successes, failures and their relative credit. Cross-model/tokenizer mismatch needs sequence compatibility and token clipping. Do not mark borrowed data on-policy.
Illustrative example: Give the receiver a peer's complete group and credit rather than inserting a single gold answer.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
Why use average sequence likelihood for cross-tokenizer GRAFT compatibility?
Learning Beyond What You Sample: Off-Policy-Aware Cross-Model Trajectory Exchange for RLVR
Show answer and example
Answer: Token counts and boundaries differ, so use sequence-level compatibility then ratio clipping to bound updates. Track text rescoring, provenance and group advantages. This doesn't make policies identical.
Illustrative example: The same sentence tokenized differently cannot be compared by subtracting probabilities position by position.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
Why borrow a complete peer group rather than one success?
Learning Beyond What You Sample: Off-Policy-Aware Cross-Model Trajectory Exchange for RLVR
Show answer and example
Answer: It retains internal advantages. Adding only a positive changes the group distribution and credit baseline. Borrowed trajectories are not receiver on-policy data.
Illustrative example: Keep peer failures to preserve why peer successes have relative positive advantage.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
When does GRAFT borrow a peer group?
Learning Beyond What You Sample: Off-Policy-Aware Cross-Model Trajectory Exchange for RLVR
Show answer and example
Answer: When receiver candidates all fail and a peer group for the same prompt contains both success and failure. Only then is there transferable relative credit; it is not copying one answer from the best model.
Illustrative example: Four receiver failures can use a peer group with both correct and incorrect replies.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Core mechanism
Why lag GATS success and permanently withdraw after catching teacher?
Guide, Then Let Go: Gap-Adaptive Teacher Scheduling for Sparse-Reward Agentic RL
Show answer and example
Answer: Lag avoids same-batch feedback and fixed benchmark stabilizes comparison. Linearly reduce distillation as gap closes, then keep teacher off despite fluctuations. Noise remains and only state machine was validated.
Illustrative example: An improvement affects guidance next batch; reaching benchmark isn't a reversible toggle.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Implementation details
Why use one-step-lagged EMA and fixed teacher tail success?
Guide, Then Let Go: Gap-Adaptive Teacher Scheduling for Sparse-Reward Agentic RL
Show answer and example
Answer: Current rewards cannot alter their own schedule. Fixed teacher baseline defines gap; handle zero baseline. Only schedule kernel, not Qwen agent RL.
Illustrative example: Log current batch then let next batch read updated EMA.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Method comparison
How does permanent withdrawal differ from switching per batch?
Guide, Then Let Go: Gap-Adaptive Teacher Scheduling for Sparse-Reward Agentic RL
Show answer and example
Answer: Permanent state avoids oscillatory recalls; dynamic switching can restore help after a drop. Noisy pass rates mean reaching target once isn't universal superiority.
Illustrative example: A slight later decline won't restart teacher calls by design.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Basics
Why lower GATS teacher weight as the student closes the gap?
Guide, Then Let Go: Gap-Adaptive Teacher Scheduling for Sparse-Reward Agentic RL
Show answer and example
Answer: Teacher gives direction early under sparse rewards, but near a fixed reference it can restrict independent exploration. Permanently withdraw at the target and continue reward training.
Illustrative example: After reaching teacher-level success, continued imitation may hinder improvements.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Core mechanism
Why freeze rollout states for kernel gradients?
FlowMap-OPD: Rollout-Kernel Separation for On-Policy Distillation of Few-Step Flow-Map Generators
Show answer and example
Answer: Rollout supplies required marginals; kernel compares flows or velocity at those states. Differentiating through generation adds unspecified trajectory terms. Detached states can be recollected from later policies, not permanently fixed.
Illustrative example: Sample intermediate states, freeze them for one local teacher comparison, and don't backpropagate through entire sampling.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Implementation details
Does local FlowMap train a real image generator?
FlowMap-OPD: Rollout-Kernel Separation for On-Policy Distillation of Few-Step Flow-Map Generators
Show answer and example
Answer: No. It verifies state detach and local KL. Freeze teacher; caller ensures required rollout marginals. Random vectors need not follow the paper's distribution.
Illustrative example: Computing KL for an arbitrary vector doesn't validate generation provenance.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Method comparison
How does detaching states differ from end-to-end differentiation through the sampler?
FlowMap-OPD: Rollout-Kernel Separation for On-Policy Distillation of Few-Step Flow-Map Generators
Show answer and example
Answer: Detach trains student kernel parameters only. Differentiating through states adds dependency gradients and changes the estimator despite equal scalar loss.
Illustrative example: Current model may affect both states and predictions; freezing states deliberately cuts the former path.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Basics
Why separate rollout distribution and optimization kernel in FlowMap-OPD?
FlowMap-OPD: Rollout-Kernel Separation for On-Policy Distillation of Few-Step Flow-Map Generators
Show answer and example
Answer: Sampling supplies states with specified marginals; a local kernel compares teacher/student flows or velocities. Separate data provision from gradients rather than backpropagating through sampling arbitrarily.
Illustrative example: Freeze a sampled intermediate state, then compare local predictions.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Core mechanism
Why add future potential to current chunk reverse KL?
DriftOPD: Sequence-Level Reverse-KL Distillation for One-Step VLA Policies
Show answer and example
Answer: Immediate matching cannot capture state transitions. A demonstration-trained critic approximates long-horizon effects to retain timing in sequential decomposition. Coverage/critic accuracy matter; no local environment validation.
Illustrative example: Two grabs look teacher-like now, but one puts the object out of reach later.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Implementation details
Does local DriftOPD execute a robotics closed loop?
DriftOPD: Sequence-Level Reverse-KL Distillation for One-Step VLA Policies
Show answer and example
Answer: No. It verifies objective decomposition and teacher detach. Real work needs an offline-trained future critic, chunk/state alignment and environment evaluation.
Illustrative example: A computed potential term does not show a robot completed the task.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Method comparison
How does future potential differ from an immediate reward?
DriftOPD: Sequence-Level Reverse-KL Distillation for One-Step VLA Policies
Show answer and example
Answer: It estimates downstream outcomes within a sequence decomposition; immediate reward rates current behavior. Critic approximation matters and adding value is not exact sequence KL.
Illustrative example: Lifting the object earns immediate credit even if placement becomes impossible.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Basics
Why add future potential beyond current chunk distillation in DriftOPD?
DriftOPD: Sequence-Level Reverse-KL Distillation for One-Step VLA Policies
Show answer and example
Answer: Actions alter future robot states. One-step matching misses downstream effects. Sequential reverse KL includes a future term estimated from offline demonstration critic.
Illustrative example: A plausible grasp can prevent later placement.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Core mechanism
Why weight by downstream reward JVP rather than disagreement size alone?
Dr. OPD: Learning What to Follow for Optimal On-Policy Distillation of Large Language Models
Show answer and example
Answer: Large gaps may be stylistic. JVP estimates reward sensitivity along correction, then closed-form weights are bounded and normalized for OPD. Local reward and approximation do not prove global causal contribution.
Illustrative example: Teacher and student can disagree on an operator and politeness, but only one affects task reward.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Implementation details
What order should Dr. OPD use for weights and student updates?
Dr. OPD: Learning What to Follow for Optimal On-Policy Distillation of Large Language Models
Show answer and example
Answer: Solve weights from fixed batch signals, clip and mean-normalize, then update student by weighted RKL. Do not differentiate the student through weight selection. Local d×g is a proxy.
Illustrative example: Choose supervision positions before the student step rather than changing the selector with that step.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Method comparison
How does reward JVP weighting differ from ranking by KL disagreement?
Dr. OPD: Learning What to Follow for Optimal On-Policy Distillation of Large Language Models
Show answer and example
Answer: Disagreement measures distance; JVP estimates downstream reward change along a direction. Combine gap and value, but a local gradient isn't independent correctness evidence.
Illustrative example: A large stylistic disagreement can have weak reward effect.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Basics
Why does Dr. OPD give teacher tokens different weights?
Dr. OPD: Learning What to Follow for Optimal On-Policy Distillation of Large Language Models
Show answer and example
Answer: Fixing key reasoning matters more than rephrasing. A bilevel approximation weights by expected post-update student reward, not just disagreement.
Illustrative example: At equal probability gap, correcting an operator may matter more than changing a connective.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Core mechanism
Why score a fixed executor reply in both advice conditions?
AdviSD: Learning to Advise Frontier LLMs via Targeted Multi-Turn Self-Distillation
Show answer and example
Answer: Fixed reply controls generation randomness and isolates preference shift. Train advisor on threshold-passing reflections while executor stays frozen. A likelihood increase isn't task success and calibration depends on advice distribution.
Illustrative example: Rescore the same recorded tool call in both contexts rather than subtract independently generated answers.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Implementation details
May AdviSD generate separate replies for with/without advice comparisons?
AdviSD: Learning to Advise Frontier LLMs via Targeted Multi-Turn Self-Distillation
Show answer and example
Answer: Not in place of paired rescoring. Use the same response and mean valid-token logp; threshold from donor absolute contrast quantile. No local real executor.
Illustrative example: Different newly generated long and short answers confound advice effects.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Method comparison
Is influential advice necessarily successful advice?
AdviSD: Learning to Advise Frontier LLMs via Targeted Multi-Turn Self-Distillation
Show answer and example
Answer: No. Likelihood contrast measures preference shifts on one recorded response. Donor calibration screens weak effects, not whether the change improves outcomes; evaluate tasks separately.
Illustrative example: Advice can increase support for an existing wrong action.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Basics
Why train AdviSD advisor and freeze executor?
AdviSD: Learning to Advise Frontier LLMs via Targeted Multi-Turn Self-Distillation
Show answer and example
Answer: Improve advice to a fixed strong executor, isolating advisor contribution. Reflection generates candidate advice; only advice affecting executor preferences enough enters self-distillation.
Illustrative example: Improve parameter-check reminders without changing the tool model weights.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Core mechanism
How does reading predicted future latents differ from reading actual future frames?
V-JEPA Policy: Building Effective World-Action Models on Predictive Visual Latents
Show answer and example
Answer: Predictions come from current observations and instructions and are available at deployment. Real future is training-only supervision. The action expert reads computable foresight, not actual outcomes. Frozen visual encoders do not prevent forecast errors from accumulating.
Illustrative example: Predict a cup's position after grasping; do not feed the real post-grasp photo before judging policy success.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Implementation details
How are future-target gradients isolated, and what is locally implemented?
V-JEPA Policy: Building Effective World-Action Models on Predictive Visual Latents
Show answer and example
Answer: Encode actual future with frozen vision and stop gradients. Apply masked L1 and train action velocity. Local work checks future-latent isolation and flow only; no V-JEPA 2.1 checkpoint or robot dataset. Three-seed small tensors do not validate a LIBERO policy.
Illustrative example: No target gradient confirms supervision isolation, not robot success without rollouts.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Method comparison
How does joint world/action training differ from separately training a predictor and action head?
V-JEPA Policy: Building Effective World-Action Models on Predictive Visual Latents
Show answer and example
Answer: Joint training coordinates future context and action flow so predictions can serve control needs. Independent stages need not align. The vision encoder stays frozen; joint predictor/action training does not update every visual parameter.
Illustrative example: Future grasp context can adapt to the action head while the V-JEPA encoder weights remain unchanged.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Basics
Why condition robot actions on what the model expects to see?
V-JEPA Policy: Building Effective World-Action Models on Predictive Visual Latents
Show answer and example
Answer: Current images alone may not indicate how the goal changes the scene. V-JEPA Policy connects instruction-conditioned prediction with action learning, giving the action expert predicted context. No future ground truth enters control, and actual execution must still be tested.
Illustrative example: Predict the cup inside a box without reading an actual completion image before the robot moves.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Core mechanism
How does a second key change Triadic Linear Attention?
Triadic Linear Attention: Three-Dimensional Recurrent States for Long-Context Sequence Modeling
Show answer and example
Answer: It adds an independently addressed key axis to ordinary key–value state and contracts with two queries. Capacity grows with that dimension, not for free. E=1 should recover ordinary linear attention.
Illustrative example: A person's value can depend on the scene as a second address, at the cost of a larger joint state.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Implementation details
Why test the E=1 limit instead of merely checking forward execution?
Triadic Linear Attention: Three-Dimensional Recurrent States for Long-Context Sequence Modeling
Show answer and example
Answer: It must exactly reduce to ordinary linear attention, testing construction and contraction of the extra axis. Also compare explicit history sums, causality, and gradients; this is not long-book language quality.
Illustrative example: Remove the second key's freedom and match the same-input two-dimensional reference.
Sections in the Chinese source: 背景与主要改动, 本地复现. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Method comparison
How is a second key axis different from widening values?
Triadic Linear Attention: Three-Dimensional Recurrent States for Long-Context Sequence Modeling
Show answer and example
Answer: Dimension E increases state capacity by E and enables an independent query contraction. Wider values add output content but not dual-key addressing.
Illustrative example: Adding more fields to records is different from adding a search key.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Basics
Is linear-attention state a history list or accumulated associative memory?
Triadic Linear Attention: Three-Dimensional Recurrent States for Long-Context Sequence Modeling
Show answer and example
Answer: It accumulates key/value outer products into a matrix queried by contraction. Triadic adds another key axis, not a raw per-token KV list.
Illustrative example: Multiple writes combine in a table with an added address dimension.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Core mechanism
Why gate corrections to TADM cached anchors based on current diffusion state?
Time-Anchored Diffusion Language Models: Latent-Space Caching for Fast Generation
Show answer and example
Answer: Reusing expensive latent outputs saves anchor calls, but changing states make anchors stale. A cheap conditioned fusion predicts and gates corrections for a smaller denoiser. It cannot guarantee full recovery, and local tests omit full generation.
Illustrative example: Adjust an early rough sentence representation once later words become known.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
What boundary checks validate TADM's g and Δ fusion?
Time-Anchored Diffusion Language Models: Latent-Space Caching for Fast Generation
Show answer and example
Answer: Use h_t′+g⊙Δ. Zero g or zero Δ recovers the anchor, not zero. Local cache correction does not load DiffusionGemma-26B.
Illustrative example: Turning the gate off keeps the original anchor.
Sections in the Chinese source: 核心公式, 复现边界. The example illustrates the mechanism.
Method comparison
How does gated stale-cache correction differ from reuse unchanged?
Time-Anchored Diffusion Language Models: Latent-Space Caching for Fast Generation
Show answer and example
Answer: It combines c_t and the old anchor to adjust stale information. Refresh may still be needed; fusion is not an exact substitute for every anchor call.
Illustrative example: A changed denoising state can revise old features instead of treating them as fresh.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why needn't the expensive anchor run at every step?
Time-Anchored Diffusion Language Models: Latent-Space Caching for Fast Generation
Show answer and example
Answer: Neighboring reverse steps can reuse part of the state. Refresh periodically and use cheap state-conditioned corrections in between before smaller denoising.
Illustrative example: Several steps share one anchor but produce different corrections.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Core mechanism
Why allocate STEPQuant bits by remaining state lifetime and key-row impact?
STEPQuant: When and Where Errors Matter in Delta-Rule Recurrent State Quantization
Show answer and example
Answer: Recurrence propagates errors through time, and directions affect readout differently. Allocate block precision under average bits and fit row/column scales. Local budget/error diagnostics are not throughput or renamed uniform INT8.
Illustrative example: A small direction that survives many steps may deserve more precision than a large one that soon vanishes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Implementation details
Why verify actual average bits instead of the fraction of high-precision blocks?
STEPQuant: When and Where Errors Matter in Delta-Rule Recurrent State Quantization
Show answer and example
Answer: Width and block size both determine budget. Minimize weighted error at the actual average, and keep key-row/value-column broadcasting distinct.
Illustrative example: Average 4-bit and 8-bit blocks by element counts, not block count alone.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Method comparison
How does spatiotemporal allocation differ from uniform INT8?
STEPQuant: When and Where Errors Matter in Delta-Rule Recurrent State Quantization
Show answer and example
Answer: Allocate more bits to high-risk blocks within a fixed average and use key-row/value-column scaling. Uniform INT8 ignores lifetime and position.
Illustrative example: A long-lived, high-impact block can get more bits than a quickly forgotten one.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Basics
Why quantize fixed-size recurrent state instead of only growing KV?
STEPQuant: When and Where Errors Matter in Delta-Rule Recurrent State Quantization
Show answer and example
Answer: Each request holds state and repeatedly updates it; concurrency makes fixed state costly and errors can persist. Consider lifetime and spatial block influence.
Illustrative example: Small state across many requests consumes memory, and early errors can alter later outputs.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Core mechanism
Why amortize SPLASH transitions over remaining generation steps?
SPLASH: Switching Parallel Layouts of Attention with Seamless Handoff for LLM Serving
Show answer and example
Answer: Layouts distribute attention weights and KV ownership differently, so switching incurs transfer/handoff. Remaining savings must repay that cost under memory limits. Local work is a planner, not CUDA handoff or serving evidence.
Illustrative example: Switching can be slower with a few tokens left and useful on a long request.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Implementation details
How distinguish a transition-aware plan from executing a switch?
SPLASH: Switching Parallel Layouts of Attention with Seamless Handoff for LLM Serving
Show answer and example
Answer: The planner models memory and switch cost. The system must move weights/KV and preserve request state. No local CUDA handoff or B200/H200 performance evidence.
Illustrative example: Printing “switch to DOP” does not move cache ownership.
Sections in the Chinese source: 本地复现, 复现边界. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Method comparison
How do DOP and TP differ in simplified KV memory?
SPLASH: Switching Parallel Layouts of Attention with Seamless Handoff for LLM Serving
Show answer and example
Answer: TP uses W_A/T+Bks; DOP uses W_A/T+Bks/T. Both shard weights but distribute KV differently. These are capacity estimates, not measured throughput.
Illustrative example: A dominant KV term makes distribution differences more visible.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Basics
Why model attention weights and KV ownership together?
SPLASH: Switching Parallel Layouts of Attention with Seamless Handoff for LLM Serving
Show answer and example
Answer: TP/DP/CP/DOP assign weights, requests, and caches differently as batches and sequence lengths vary. One ownership model compares memory, communication, and switching.
Illustrative example: Four devices have different per-device memory if each request's KV is owned or sharded.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why propagate gradients across several PUMBA carry steps?
On Trajectory-Aware Training for Masked Diffusion Language Models
Show answer and example
Answer: Carry affects future denoising, so isolated loss may not learn useful carry. Unroll on-policy reveals and backpropagate within fixed windows, truncating between windows for cost. Local work is not full diffusion LM training.
Illustrative example: Later fill-in errors should shape how earlier carry represents entity constraints.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Should PUMBA detach carry every step or at window boundaries?
On Trajectory-Aware Training for Masked Diffusion Language Models
Show answer and example
Answer: Keep gradients through the fixed window and detach between windows. Per-step detachment removes the mechanism; fully untruncated training changes memory and protocol. Local reference did not train LLaDA-8B.
Illustrative example: At window length three, step three reaches two earlier steps in the same window only.
Sections in the Chinese source: 核心公式, 复现边界. The example illustrates the mechanism.
Method comparison
How does trajectory-aware carry training differ from independent steps?
On Trajectory-Aware Training for Masked Diffusion Language Models
Show answer and example
Answer: BPTT lets later loss affect earlier carry. Independent examples cut those dependencies, and summing their CEs does not restore them.
Illustrative example: Step-two loss should influence step-one carry, not only local step-two computation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why train on the model's own progressive-unmasking trajectory?
On Trajectory-Aware Training for Masked Diffusion Language Models
Show answer and example
Answer: Independent noise may miss states reached by actual multistep sampling. Carry through self-generated unrolls exposes dependent decisions.
Illustrative example: Revealed words determine the next input rather than independently recreating noise.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Core mechanism
Why predict next token and next continuous state in LIFT, and where does inference state come from?
Pretraining Latent Information Feedback Transformers with Teacher Supervision
Show answer and example
Answer: Training turns the teacher's top-k next-token distribution into a continuous input and jointly supervises token and state. This relays deeper signals beyond one token. Inference feeds back the model's own state, not the teacher's future distribution.
Illustrative example: A sampled word can be accompanied by other likely alternatives in state; test accumulation of model feedback errors.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Implementation details
Can the student loss update LIFT teacher state?
Pretraining Latent Information Feedback Transformers with Teacher Supervision
Show answer and example
Answer: No. Stop teacher gradients; train with token CE and forward KL on the normalized teacher top-k distribution. Inference must use the model's own state.
Illustrative example: Student errors update the student, not teacher targets to match its mistakes.
Sections in the Chinese source: 背景与主要改动, 本地复现. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Method comparison
How does LIFT state feedback differ from returning only the sampled token?
Pretraining Latent Information Feedback Transformers with Teacher Supervision
Show answer and example
Answer: A token preserves one discrete choice. State carries continuous top-k distribution information through the shared embedding space. Dedicated state supervision remains necessary; adding any extra vector is not equivalent.
Illustrative example: Two nearly tied options contain information a single sampled word loses.
Sections in the Chinese source: 背景与主要改动, 本地复现. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Basics
What carries deep information back to the next shallow step in LIFT?
Pretraining Latent Information Feedback Transformers with Teacher Supervision
Show answer and example
Answer: A continuous state accompanies the next discrete token. Training uses teacher top-k states and predicts token and next state; inference uses self-predicted states.
Illustrative example: Pass a richer distribution alongside the generated word.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Core mechanism
Why buffer high-precision updates in a window instead of quantizing each step?
LeapQuant: Efficient Linear Attention with Accurate Recurrent State Quantization
Show answer and example
Answer: Repeated rounding feeds back into recurrence. Keep a low-bit base plus precise increments and requantize at boundaries; separate compensation protects outlier directions. Buffer and compensation cost memory. Local truncated SVD is not the original kernel or throughput.
Illustrative example: Accumulate tiny updates before rounding so each is not lost individually.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Implementation details
Should the low-bit part quantize S or S−C?
LeapQuant: Efficient Linear Attention with Accurate Recurrent State Quantization
Show answer and example
Answer: Form C from truncated SVD, quantize S−C, then add C at readout. Quantizing full S and adding C counts it twice. Merge precise window updates at the prescribed boundaries.
Illustrative example: An outlier direction stored in C need not be stored in full in the residual too.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Method comparison
What is the trade-off between window buffers and low-bit writes every step?
LeapQuant: Efficient Linear Attention with Accurate Recurrent State Quantization
Show answer and example
Answer: Buffers add temporary high-precision state but reduce whole-state requantization; stepwise writes round more often. Count base, updates, and compensation.
Illustrative example: Buffering eight updates before one quantization differs from eight whole-state writes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Basics
Why address both repeated rounding and state outliers?
LeapQuant: Efficient Linear Attention with Accurate Recurrent State Quantization
Show answer and example
Answer: Recurrence accumulates quantization error and outliers dominate scale. Window buffering reduces rounding frequency; high-precision compensation separates large directions before residual quantization.
Illustrative example: Do not recompress every token or let one direction consume the entire grid.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Structured Decisions / Typed Readout: From Generated Sentences to Candidate Distributions
Core mechanism
Why score Chinese-Jev candidate markers rather than generate option names and parse them?
Chinese-Jev: Bringing System One Model to Chinese-Language Tasks
Show answer and example
Answer: Closed decisions need a distribution across candidates. Encode context and candidates, score each marker's hidden state and normalize, avoiding free generation and parsing. Soft targets retain annotation probabilities. A candidate's share depends on its set and is not independent correctness probability; the local objective does not reproduce general pretraining.
Illustrative example: Example: A ticket receives refund, shipping and account probabilities. Adding a category changes the shares; the old refund probability isn't fixed confidence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Structured Decisions / Typed Readout: From Generated Sentences to Candidate Distributions
Implementation details
Which axis should Chinese-Jev softmax use?
Chinese-Jev: Bringing System One Model to Chinese-Language Tasks
Show answer and example
Answer: Normalize over valid candidates within one request. Score marker states with the same function. Do not share a denominator across requests; align soft-label order to candidates.
Illustrative example: Example: Two tickets with three categories each require two distributions, not one over six scores.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Structured Decisions / Typed Readout: From Generated Sentences to Candidate Distributions
Method comparison
Do one-hot and soft labels give Chinese-Jev the same supervision?
Chinese-Jev: Bringing System One Model to Chinese-Language Tasks
Show answer and example
Answer: No. One-hot keeps a single target; soft distributions preserve probability allocation across choices, reflecting ambiguity or annotator differences. The readout is shared, not the supervision. Soft labels need trustworthy provenance and are not true accuracy by virtue of precision.
Illustrative example: Example: Labels 0.6 and 0.4 retain a close preference; 1 and 0 lose it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Structured Decisions / Typed Readout: From Generated Sentences to Candidate Distributions
Basics
What output tasks suit Chinese-Jev?
Chinese-Jev: Bringing System One Model to Chinese-Language Tasks
Show answer and example
Answer: Finite-choice decisions such as classification, routing and candidate selection. It takes Chinese context and candidates and directly returns candidate probabilities rather than generating long free-form answers.
Illustrative example: Example: Route an inquiry to shipping or refunds from a supplied list; writing an open-ended article is a different interface.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Core mechanism
Why predict expert error in addition to affinity?
Cross-Entropy Guided Routing in Mixture-of-Experts Large Language Models
Show answer and example
Answer: Affinity need not reflect expected next-token loss. An error head attenuates high-predicted-error logits before top-k and learns from actual next-token CE. Route using predictions available before the answer, not true CE.
Illustrative example: Between similar experts, predicted error can promote the more suitable one without seeing the true next token online.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Implementation details
Is applying the error penalty after top-k equivalent?
Cross-Entropy Guided Routing in Mixture-of-Experts Large Language Models
Show answer and example
Answer: No. Pre-softmax correction can change which experts are selected. Post-selection only reweights winners. Use true next-token CE to supervise the error head, not arbitrary heuristics.
Illustrative example: A third-ranked expert can replace a high-error first choice only before the selection is final.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Method comparison
How does error-aware routing differ from load balancing?
Cross-Entropy Guided Routing in Mixture-of-Experts Large Language Models
Show answer and example
Answer: Error-aware routing seeks token quality; balancing allocates compute evenly. Neither objective automatically supplies the other.
Illustrative example: Equal token counts can still send difficult cases to unsuitable experts.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Basics
What do affinity and token error express?
Cross-Entropy Guided Routing in Mixture-of-Experts Large Language Models
Show answer and example
Answer: Affinity is original router preference, while predicted error estimates quality for this token. Correct logits before top-k and supervise the head with actual next-token CE.
Illustrative example: A strongly matched expert can still be bad at that token type.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Core mechanism
Why can reading only a failed trace be insufficient to diagnose evidence collection in a video agent?
Video-RSI: Recursive Self-Improvement of Video Understanding Agents via Harness Evolution
Show answer and example
Answer: A trace records only what the old harness chose to observe; unseen keyframes cannot be recovered from that log. Revisit training video to gather observations and test whether evidence was omitted or reasoning was wrong, then change the interface. Acceptance retains accuracy and cost gates so a weighted total cannot hide a clear regression in either.
Illustrative example: If the old policy sampled only the opening while the key action came at the end, the trace omits it. Revisiting the original training video verifies that sampling rather than wording needs to change.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Implementation details
Why isn't a passing local Pareto gate registered as an executable Evolve operator?
Video-RSI: Recursive Self-Improvement of Video Understanding Agents via Harness Evolution
Show answer and example
Answer: There is no real video decoding, code-rewriting sandbox, or cross-generation stopping condition, so the controller cannot fully execute and evaluate a candidate. The gate only checks acceptance relations over supplied metrics. Fixture scores do not establish video-agent improvement, and a registry name or three-seed rule check is not end-to-end evolution.
Illustrative example: Two sets of numbers suffice to calculate dominance, but without obtaining them by running a candidate harness on video there is no capability evidence for promotion.
Sections in the Chinese source: 核心公式, 本地复现与边界. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Method comparison
What do dual accuracy and cost gates protect compared with a single weighted score?
Video-RSI: Recursive Self-Improvement of Video Understanding Agents via Harness Evolution
Show answer and example
Answer: A candidate must not lower accuracy or increase cost, and at least one measure must strictly improve. This prevents large gains on one objective from hiding deterioration on the other. Declare tolerances before selection rather than relaxing them to accept a candidate. Dual gates still require the same measurement protocol and do not guarantee generalization.
Illustrative example: A candidate that loses ten accuracy points while halving frame count could win a weighted sum but violates the no-regression requirement and should be rejected.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Basics
Why might a video agent need to improve evidence collection rather than just use a stronger answer prompt?
Video-RSI: Recursive Self-Improvement of Video Understanding Agents via Harness Evolution
Show answer and example
Answer: Long video spans many moments and details. The failure may occur during frame selection, localization, reading, or organizing evidence, rather than composing the final answer. Video RSI uses execution to improve the video-task harness so limited cost is spent on relevant evidence. A changed workflow alone is not better understanding; check both accuracy and actual cost.
Illustrative example: If the question asks what someone did before leaving and the agent samples only the start and end, rewording the final prompt will not reveal the key middle action. Improve evidence gathering first.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Core mechanism
Why can premature disclosure by a user simulator improve task success but reduce UFS?
UserProxyBench: Evaluating LLM User Simulators for Agent Benchmarks and Training
Show answer and example
Answer: The agent should ask for information, and unsolicited disclosure bypasses that capability. UFS separately checks every applicable user contract per episode rather than reusing task reward. Completion and fidelity are distinct; success alone does not establish simulator quality.
Illustrative example: Giving an order number before being asked may make refunds succeed without testing whether the agent knew when to request it.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Implementation details
Why retain the order of request and disclosure when auditing premature disclosure?
UserProxyBench: Evaluating LLM User Simulators for Agent Benchmarks and Training
Show answer and example
Answer: Prematurity depends on whether the relevant request came first, not whether a field eventually appears. Local evaluation checks public interaction events against the rubric while keeping private blueprints on the grading side. It does not run paper enterprise tasks or the closed-proxy matrix.
Illustrative example: Disclosing the same field before or after a question can leave identical final text but different fidelity verdicts.
Sections in the Chinese source: 背景与主要改动, 本地复现. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Method comparison
How does the UFS all-criteria product differ from averaging criterion pass rates?
UserProxyBench: Evaluating LLM User Simulators for Agent Benchmarks and Training
Show answer and example
Answer: An episode scores one only when all applicable criteria pass; any failure gives zero. Per-criterion means can let other passes offset a key violation and are a different metric. Report UFS separately from task reward.
Illustrative example: Passing nine of ten criteria gives 90% per-criterion average but strict UFS zero for that episode.
Sections in the Chinese source: 背景与主要改动, 本地复现. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Basics
Why does UserProxyBench separate simulator fidelity from task reward?
UserProxyBench: Evaluating LLM User Simulators for Agent Benchmarks and Training
Show answer and example
Answer: Simulators can leak private fields and let agents succeed with less information collection. Independent UFS checks every user contract, avoiding misreading reward on an easier task as real interaction ability.
Illustrative example: An unrequested hidden order number can enable success without showing strong clarification ability.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why isn't the utility of a memory set the sum of individual relevance scores?
UpliftMem: Learning Set-Level Uplift for Agent Memory Retrieval
Show answer and example
Answer: Items can complement, duplicate, or conflict, so set-level execution effects are not additive relevance. The method supervises on the difference with and without memory for the same frozen executor. Additional rollouts go where they can change the local choice. Relevance proposes candidates, not measured uplift.
Illustrative example: Example: One item gives an API name and another its argument constraints; together they enable execution. Two similar copies of the API introduction may be less useful.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
Why does set_level_uplift require paired treatment and control?
UpliftMem: Learning Set-Level Uplift for Agent Memory Retrieval
Show answer and example
Answer: Compare memory versus no memory on the same task, frozen executor, and matched conditions to reduce task and model confounds. The local run implements paired differences and EVSI on a fixed posterior, not full set construction, shared scoring, or real execution. Component outputs are not agent success gains.
Illustrative example: Example: A memory set on a hard problem versus no memory on an easy one is not comparable. Both conditions must face the same instance.
Sections in the Chinese source: 本地复现. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
How does EVSI allocation differ from giving each candidate the same number of extra rollouts?
UpliftMem: Learning Set-Level Uplift for Agent Memory Retrieval
Show answer and example
Answer: Uniform allocation ignores whether uncertainty can change the selection. EVSI asks how much an observation is expected to change the local retrieval decision. A correlated Gaussian posterior expresses candidate dependencies. This allocates budget; the highest predicted mean is not observed success.
Illustrative example: Example: Extra probing is low-value when two sets are far apart. A rollout can be more useful for two close, uncertain alternatives.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Why does UpliftMem distinguish relevant retrieval from helping the agent complete a task?
UpliftMem: Learning Set-Level Uplift for Agent Memory Retrieval
Show answer and example
Answer: Similar memories can duplicate what is known or conflict with other records. Relevance scores need not measure execution value. UpliftMem asks whether a complete set gives a fixed executor additional benefit, reserving context for evidence combinations that change outcomes.
Illustrative example: Example: Three high-similarity records give the right API name but omit authentication. The task still fails, so similarity alone does not establish utility.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why improve both item perception and bounded memory in a recommendation agent instead of only shortening history?
ReMem: Rethinking Perception and Memory in Long-Context Recommendation Agents
Show answer and example
Answer: Bad page extraction makes even a sophisticated preference memory retain wrong semantics. OCR perception first produces compact item representations, then dynamic memory updates preferences under budget. Several memory views share policy training while supplying distinct evidence. Test preference answers must never be injected to create apparent gains.
Illustrative example: Example: A price in small print on an image cannot be corrected by compressing browsing history. Extract reliable attributes first, then retain information relevant to the user's budget.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
Why can't Multi-Memory GRPO mix all memory views into one advantage group?
ReMem: Rethinking Perception and Memory in Long-Context Recommendation Agents
Show answer and example
Answer: Normalize rewards separately within each memory view for the same query, then share the clipped policy update. Mixing views can confuse differences in view difficulty with action advantage. The local run tests budget, temporal updates, and objectives, not OCR, browsers, or real recommendation episodes.
Illustrative example: Example: Sparse views may have lower rewards overall. Mixing them with rich views can wrongly attribute that difficulty to particular responses.
Sections in the Chinese source: 核心公式, 本地复现与边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
What is the trade-off between dynamic fixed-budget memory and full history?
ReMem: Rethinking Perception and Memory in Long-Context Recommendation Agents
Show answer and example
Answer: Full history retains more but costs grow with interactions. Dynamic memory ranks by relevance and recency and emphasizes current preferences within budget, at the risk of losing rare important constraints. Recent observations do not replace auditing long-term facts, and real user episodes are needed to measure recommendation quality.
Illustrative example: Example: Recent running-shoe purchases should not erase a longstanding material allergy. Verify the constraint survived compression.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Why are the input challenges of long-context recommendation agents broader than long history?
ReMem: Rethinking Perception and Memory in Long-Context Recommendation Agents
Show answer and example
Answer: The agent must understand heterogeneous item pages and track preferences through accumulating user behavior. Errors on either side can lead to poor recommendations. ReMem studies perception and evolving memory together to provide enough decision evidence within a fixed budget, not simply add history.
Illustrative example: Example: Product weight is embedded in an image and the user now prefers light items. Shorter chat history alone neither reads the weight nor guarantees an updated preference.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Core mechanism
Why specify when, provide, and use in a meta-skill rather than just 'verify more in future'?
Learning Meta-Skills for Agent Harness Design in Test-Time AI4AI
Show answer and example
Answer: When gives the trigger, provide specifies the resource, and use describes integration. Together they turn generic advice into retrievable design knowledge. Freeze the library after development and retrieve only to construct the harness in testing. If test results revise the library, the test no longer measures transfer of fixed skills to unseen tasks.
Illustrative example: 'When tool output is incomplete, provide access to raw logs and inspect them before retrying' can become a concrete runtime component more readily than 'be more careful.'
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Implementation details
How do local development refinement and frozen retrieval limit the conclusions?
Learning Meta-Skills for Agent Harness Design in Test-Time AI4AI
Show answer and example
Answer: Local checks verify that the library accepts development feedback, freezes against write-back during retrieval, and maintains related state invariants. Fixture rewards are not the AI4AI benchmark. Without a real harness proposer and task model, matching a when field neither establishes generalizable design learning nor reproduces reported paper gains.
Illustrative example: Finding a meta-skill tagged 'long text' tests routing. It does not test whether the skill improves real long-context tasks.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Method comparison
How does frozen-library retrieval during testing differ from writing experience back while testing?
Learning Meta-Skills for Agent Harness Design in Test-Time AI4AI
Show answer and example
Answer: Freezing tests transfer from development. Writing back from test feedback lets later tasks exploit evaluation results and changes the test information contract. Allowed observations in the current task can inform its harness, but private test scores cannot revise the long-term library or reselect reported results.
Illustrative example: Adding a rule tailored to a failed test1 and rerunning test1 is not frozen-library generalization. Report it under a separate learning protocol.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Basics
Why does Meta-Skills focus on transferable experience about configuring how an agent works?
Learning Meta-Skills for Agent Harness Design in Test-Time AI4AI
Show answer and example
Answer: Similar tasks can share planning, evidence collection, or verification routines. Keeping only individual answers forces the next task to redesign the workflow. Meta-Skills makes procedural experience reusable through retrieval that guides the harness. Experience needs applicability conditions; a prompt that helped one question should not be applied everywhere.
Illustrative example: Building an evidence table before comparing numbers across sources is procedural knowledge. The actual numbers belong to the original task and should not be reused as new answers.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Core mechanism
Why does the controller use a compact state while keeping full results in artifact memory?
Thinking Before Thinking: Scaling Agentic Inference Through Meta-Reasoning
Show answer and example
Answer: Control decisions need progress, candidate actions, and remaining budget. Repeating full material in every prompt raises cost. Persistent artifacts hold reusable evidence for on-demand reference. Assess, Propose, Evaluate, and Dispatch divide responsibility for continuing or branching. Compressing control state must not discard actual work.
Illustrative example: Example: Two workers each write an analysis. The controller keeps the conclusions and file references, then opens the original artifact when needed instead of repeating it in every turn.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Implementation details
How does the local selector weigh expected value and call cost, and what remains unverified?
Thinking Before Thinking: Scaling Agentic Inference Through Meta-Reasoning
Show answer and example
Answer: It rejects actions that exceed the budget, then selects using expected value and call cost. Full artifacts stay in persistent memory while control state stays compact. Expected values are explicit inputs, not success predictions learned from frontier workers. Three-seed diagnostics verify scheduling and state separation only.
Illustrative example: Example: With two calls left, a three-call action is infeasible regardless of expected value. Do not dispatch it and hide the budget overrun later.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Method comparison
What are the trade-offs of separate Assess, Propose, Evaluate, and Dispatch steps versus a single direct controller?
Thinking Before Thinking: Scaling Agentic Inference Through Meta-Reasoning
Show answer and example
Answer: Separation clarifies state summarization, candidate generation, comparison, and execution, and makes budget feasibility independently checkable. Extra control steps cost calls and may not pay off for simple tasks. Direct control is compact but can conflate evaluation with execution. Compare all calls and final outcomes.
Illustrative example: Example: A one-step calculation may not need four rounds of control. In complex research, assessing whether to branch can avoid expensive duplicated experiments.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Basics
How does an object-level worker differ from a meta-reasoning controller?
Thinking Before Thinking: Scaling Agentic Inference Through Meta-Reasoning
Show answer and example
Answer: A worker performs task content such as retrieval, coding, or analysis. The controller assesses progress, proposes continuation or branching, compares expected benefits, and dispatches work. Control consumes model-call budget and must count in equal-budget comparisons. Persistent artifacts avoid redoing work.
Illustrative example: Example: A worker writes a test file. The controller decides whether to fix failures, try another approach, or stop. The outputs and costs differ.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why does letting a model edit a context file also change KV reuse cost?
Show answer and example
Answer: The file prefix determines whether cached representations still match. Changing early tokens usually requires re-prefilling the suffix. File transforms therefore determine both what to remember and future runtime cost. Shorter text is not always cheaper when it invalidates a large cache.
Illustrative example: Example: Appending an observation at the end can preserve prefix KV. Rewriting the opening summary can force recomputation despite fewer total words.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
Why must suffix cache reuse recompute from the first token mismatch?
Show answer and example
Answer: Causal hidden states depend on earlier tokens. After the first mismatch, later identical text can still have different KV. Keep only the longest common prefix and check file capacity explicitly. The local run tests edits and boundaries without CLM training or a long multi-agent swarm.
Illustrative example: Example: Changing token 100 can alter token 101 and all following states, not just the replaced position.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
How do context editing and direct KV eviction differ in what logical history remains?
Show answer and example
Answer: File edits change the logical token sequence the model sees and require prefix matching before reuse. Direct KV eviction can retain logical text while changing active cache state. Reducing active context in both cases does not imply the same cache identity or recovery semantics.
Illustrative example: Example: Deleting early file text changes the prefix for all later tokens, so the old full KV cannot be copied. Dropping KV alone can leave the text intact.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Why is a context file part of model behavior rather than a fixed external summary?
Show answer and example
Answer: The model chooses what to keep, delete, and reorganize; later actions read the edited state. In a fixed harness, external rules choose what and when to compress. Model editing changes responsibility and risks losing important constraints, so capacity and execution contracts still matter.
Illustrative example: Example: Delete an obsolete log and keep its conclusion proactively instead of waiting for a length threshold to summarize all history.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Core mechanism
Why remove development examples solved by every branch while retaining examples that distinguish branches?
Mixture of Self-Improving Branches for Agent Harness Optimization
Show answer and example
Answer: Shared solved examples provide little information about branch differences and can dilute weaknesses while consuming budget. Each branch retains a discriminating subset and proposal strategy, and a deployment router picks a development champion by input. An adaptive development set is not an independent final test; routing cannot use test labels to choose a winner.
Illustrative example: If all branches solve simple sorting but graph search distinguishes two strategies, devote more curriculum budget to the latter rather than repeatedly testing common strengths.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Implementation details
May a deployment router read branches' true scores on the current test item to pick the winner?
Mixture of Self-Improving Branches for Agent Harness Optimization
Show answer and example
Answer: No. Use input features to select a development champion without test labels or future execution scores. The local implementation provides subset diversification and linear routing, not LLM-generated harnesses or SWE-bench runs. Correct branch choice on fixtures does not show real mixture gains.
Illustrative example: Running all branches and selecting the hidden judge's best result is an oracle ensemble, not deployment routing based on request inputs.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Method comparison
What assumption differs between routing across specialist branches and always running one global champion?
Mixture of Self-Improving Branches for Agent Harness Optimization
Show answer and example
Answer: A global champion assumes one harness covers all inputs. Branch mixtures allow workflows to specialize and choose among them from public input features. Specialization may improve fit but adds routing errors and maintenance cost. Routing must use deployment-visible information, not current test gold scores.
Illustrative example: Code and browsing tasks can have different development champions. Routing by task features is deployable; reading both branches' gold scores on the current test before choosing is not.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Basics
Why should branches maintain different development subsets and proposal policies?
Mixture of Self-Improving Branches for Agent Harness Optimization
Show answer and example
Answer: When all branches see the same examples and use the same proposals, they can converge to redundant harnesses. Distinct subsets and strategies preserve specialization that the router can select by input. Additional branches also increase development cost; do not hide total search budget when comparing with one branch.
Illustrative example: One branch can specialize in tool formats and another in long-horizon verification, giving routing meaningful alternatives rather than copies under different names.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Core mechanism
Why hold history fixed and vary only identity prompts in PromptShift, then measure Drift and SliceShift separately?
Measuring and Mitigating Identity-Cue Preference Drift in LLM-based Recommender Systems
Show answer and example
Answer: Changing both confounds actual preference and identity cues. Drift measures overlap/rank changes and SliceShift measures cohort popularity direction; mainstreamness conditions relevance and inverse-popularity blending. This is a controlled sensitivity diagnostic, not causal discrimination proof.
Illustrative example: Illustration: Use the same movie history with and without identity text to see whether rankings shift toward cohort-popular films.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Implementation details
What must stay fixed between PromptShift's reference and identity-prompt lists?
Measuring and Mitigating Identity-Cue Preference Drift in LLM-based Recommender Systems
Show answer and example
Answer: Keep history, candidates, model, and decoding as consistent as possible and vary the identity cue only. Handle shared item ranks and set changes as defined; do not pair disjoint items artificially. Do not choose mitigation on test.
Illustrative example: Illustration: If one arm also has recent clicks, drift may come from the extra behavior rather than identity.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Method comparison
Where does user conditioning enter adaptive inverse-popularity blending versus a uniform tail bonus?
Measuring and Mitigating Identity-Cue Preference Drift in LLM-based Recommender Systems
Show answer and example
Answer: Mainstreamness m changes the mix of relevance and inverse popularity, unlike a uniform bonus. Both can hurt relevance. Diagnostics do not establish real fairness or satisfaction and do not belong on formal capability leaderboards.
Illustrative example: Illustration: A mainstream user and a niche user may need different corrections; pushing even more obscurity can be unhelpful for the latter.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Basics
Does high PromptShift Drift directly imply lower accuracy?
Measuring and Mitigating Identity-Cue Preference Drift in LLM-based Recommender Systems
Show answer and example
Answer: No. Drift measures list change, not relevance of old and new items. SliceShift adds popularity direction, not quality. Independent relevance or feedback is needed; do not relabel sensitivity as accuracy loss.
Illustrative example: Illustration: Swapping two equally suitable films changes rank drift without necessarily hurting the user.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Core mechanism
Why type-check adjacent skills in a genome instead of chaining high-scoring modules arbitrarily?
EvoSkillRec: Skill-Genome Evolution for Recommender Architecture Discovery
Show answer and example
Answer: Individually good modules may have incompatible shapes and semantics. Check input/output contracts, construct executable candidates, evaluate validation, and store only promoted combinations. Names and types alone do not integrate a skill; three local seeds check composition and state transitions, not capability rankings.
Illustrative example: An item-vector producer cannot feed a full-history consumer without a suitable adapter and semantic validation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Implementation details
Why is matching final tensor width insufficient for skill type checks?
EvoSkillRec: Skill-Genome Evolution for Recommender Architecture Discovery
Show answer and example
Answer: B×d user vectors, B×C×d candidate tensors, and B×T×d sequence states have distinct roles. Check batch, candidate, and time semantics, then execute and evaluate. Local state-transition diagnostics do not rank capability.
Illustrative example: Broadcasting one user vector over all candidates can run but loses candidate differences.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Method comparison
How does validation promotion protect EvoSkillRec's reuse library versus keeping every skill?
EvoSkillRec: Skill-Genome Evolution for Recommender Architecture Discovery
Show answer and example
Answer: Only innovations exceeding a baseline threshold become reusable, reducing propagation of untested or harmful modules. The threshold is still bounded by budget and data, and neither type checks nor repeated test selection establish quality. Keep failures as failures.
Illustrative example: Record a shape-correct module that lowered validation, but do not promote it as a default skill.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Basics
Why require executable skill I/O contracts in EvoSkillRec?
EvoSkillRec: Skill-Genome Evolution for Recommender Architecture Discovery
Show answer and example
Answer: Natural-language names do not ensure composability. Types make genomes executable and uniformly evaluable but are only the first check; semantics and recommendation quality need validation and promotion separately.
Illustrative example: A sequence encoder can feed a vector head, but a candidate-set consumer needs an explicit conversion.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Core mechanism
Why retain full context but supervise selected continuation only?
ROSS: Relearning from Self-Generated Rollouts through Selective Supervision
Show answer and example
Answer: History determines what state the continuation addresses. Masked supervision avoids imitating mistakes/redundancy while keeping context visible. Fixed local masks do not implement the paper selector.
Illustrative example: An early error can remain visible while only a later successful repair is learned.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Implementation details
Is the local ROSS selector learned?
ROSS: Relearning from Self-Generated Rollouts through Selective Supervision
Show answer and example
Answer: No. A fixture mask verifies full-context/selective-loss boundaries, not autonomous selection of useful historical steps.
Illustrative example: Marking the last two sentences manually tests adherence, not discovery.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Method comparison
How does ROSS differ from deleting unselected spans before SFT?
ROSS: Relearning from Self-Generated Rollouts through Selective Supervision
Show answer and example
Answer: Deletion changes conditional state, while full context plus masked loss preserves the trace. Selector quality still matters; mask correctness is not selector implementation.
Illustrative example: Removing a wrong prefix can change meaning/likelihood of a corrective sentence.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Basics
Why reuse historical self-rollout in ROSS without imitating all of it?
ROSS: Relearning from Self-Generated Rollouts through Selective Supervision
Show answer and example
Answer: Old traces contain mistakes/redundancy. Full context conditions later behavior; selective continuation labels retain only valuable segments. Visible text need not be target.
Illustrative example: Read the failure history but supervise only its selected correction.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Core mechanism
Why supervise only retrospection tokens, not actions or feedback?
Shockingly Simple Self-retrospection Improves Agentic Models Without RL
Show answer and example
Answer: History conditions the lesson, but the target is the later reflection. Separate input and loss masks and globally average reflection tokens. Local fixtures/registry aren't a complete agent/Evolve loop.
Illustrative example: Read a bad call and error, but learn the parameter-check lesson, not the bad call or tool error text.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Implementation details
How normalize losses across differently sized ROFT retrospections?
Shockingly Simple Self-retrospection Improves Agentic Models Without RL
Show answer and example
Answer: Average all valid reflection tokens globally. Keep tasks/actions/observations/feedback as context but mask their loss. Per-reflection means change length weights. No full agent data/Evolve loop.
Illustrative example: Short and long reflections contribute different total mass under token vs example means.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Method comparison
How does ROFT differ from cloning original actions?
Shockingly Simple Self-retrospection Improves Agentic Models Without RL
Show answer and example
Answer: Cloning repeats recorded behavior; reflection supervision can teach avoiding it. Different labels and reflection quality still matters.
Illustrative example: After deleting a wrong file, learn to check paths rather than repeat deletion.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Basics
Why turn environment feedback into retrospection before ROFT learning?
Shockingly Simple Self-retrospection Improves Agentic Models Without RL
Show answer and example
Answer: Terminal reward rarely expresses reusable lessons. Text reflections summarize useful experience from successes/failures. The reflection is the target, not every historical action.
Illustrative example: A permission mistake can teach checking access before next time.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Core mechanism
Why don't OLIVE prefixes receive supervised loss?
Learning from Teacher Continuations at Student States
Show answer and example
Answer: The teacher continues the student's actual prefix, giving useful next-step examples without treating student mistakes as correct labels. Apply CE only to teacher tokens. Teacher quality and full-model experiments remain separate.
Illustrative example: Read a repairable wrong expression and learn only the corrected teacher continuation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Implementation details
Can OLIVE remove prefixes from input because they have no loss?
Learning from Teacher Continuations at Student States
Show answer and example
Answer: No. Prefix remains conditioning for continuation and student supervision; mask targets only. Handle concatenation, EOS and padding. Local tests verify masks without a closed teacher.
Illustrative example: Retain three mistaken steps as context without training to imitate them.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Method comparison
How does teacher continuation CE differ from scoring original student tokens?
Learning from Teacher Continuations at Student States
Show answer and example
Answer: Continuation adds newly generated teacher text as target; rescoring evaluates existing sampled actions/states. A scoring kernel doesn't produce new teacher continuations.
Illustrative example: Writing the next steps differs from rating the old steps.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Basics
Why let OLIVE teacher continue from student-visited states?
Learning from Teacher Continuations at Student States
Show answer and example
Answer: Student mistakes differ from ideal teacher paths. Real prefixes supply deployment-like supervision for continuation/recovery instead of memorizing complete solutions.
Illustrative example: Continue from an unusual intermediate expression the student reached.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Core mechanism
Why put conflict attribution in teacher view instead of changing every role's reward?
MAS-OPD: On-Policy Distillation for Multi-Agent Systems
Show answer and example
Answer: One team score doesn't identify corrections. Privileged teacher distinguishes target and other contributions. Extra attribution is training-only and local mask/advantage does not establish multi-model collaboration.
Illustrative example: A sound planner and an executor using the wrong tool shouldn't receive the same blame.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Implementation details
Does local MAS-OPD jointly train real models?
MAS-OPD: On-Policy Distillation for Multi-Agent Systems
Show answer and example
Answer: No. It tests role advantage and conflict masks. Align roles, traces and valid generated positions. Registry labels or misassigned signals don't establish team training.
Illustrative example: Swapping role order can keep arrays valid while assigning credit to the wrong member.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Method comparison
How does teacher-only conflict information differ from giving full attribution to all roles?
MAS-OPD: On-Policy Distillation for Multi-Agent Systems
Show answer and example
Answer: Students act on their own public histories; only teacher sees extra conflict explanation. Giving hindsight to students changes deployment inputs and hides whether learning internalized it.
Illustrative example: Teacher may know who misunderstood after failure; deployed members cannot start with that diagnosis.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Basics
Why add role-specific advantage in MAS-OPD?
MAS-OPD: On-Policy Distillation for Multi-Agent Systems
Show answer and example
Answer: Terminal team outcomes hide member responsibility. Target vs other-role teacher signals supply local collaboration credit. Verify attribution rather than punish everyone equally.
Illustrative example: Good retrieval and faulty summarization call for different credit.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Core mechanism
Why use a quadratic center with Huber tails instead of squared loss throughout?
An RL View of OPD: Least Square Policy Distillation for Sample-Efficient LLM Reasoning
Show answer and example
Answer: Quadratic shrinks small logp gaps while linear tails bound outlier influence and retain correction direction. Average responses equally. Reused old traces are not fresh current-policy samples.
Illustrative example: One teacher-improbable token can dominate squared loss; robust tails curb it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Implementation details
How handle teacher gradients and lengths in LSPD?
An RL View of OPD: Least Square Policy Distillation for Sample-Efficient LLM Reasoning
Show answer and example
Answer: Freeze teacher logp. Average differentiable student gaps over valid tokens per response, then responses equally, with specified entropy subtraction. Registry presence doesn't prove executable Evolve.
Illustrative example: Long replies do not automatically receive twice the weight.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Method comparison
How does robust LSPD behave in tails compared with reverse KL?
An RL View of OPD: Least Square Policy Distillation for Sample-Efficient LLM Reasoning
Show answer and example
Answer: It explicitly limits residual slope; KL is defined through distribution/probability weights. Clipping KL isn't equivalent. Replay also changes state provenance.
Illustrative example: Doubling tail residual changes linear and unbounded quadratic gradients differently.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Basics
Why treat teacher–student logp gaps as robust residuals in LSPD?
An RL View of OPD: Least Square Policy Distillation for Sample-Efficient LLM Reasoning
Show answer and example
Answer: Squared loss shrinks ordinary gaps; Huber-like linear tails limit gradients from extreme ones. Residual regression can reuse older trajectories.
Illustrative example: A rare token's huge log gap should not dominate a batch indefinitely.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Core mechanism
Why train multiple depth prefixes instead of truncating a finished deep model?
Show answer and example
Answer: Standard training supervises the full path, not necessarily viable intermediate exits. Telescopic trains sampled prefixes with next-token supervision plus a full-depth anchor. Evaluate shallow and deep capabilities separately.
Illustrative example: Running eight versus 24 layers is supported only if the eight-layer exit was explicitly trained.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Implementation details
How check that random prefixes do not replace the full-depth anchor?
Show answer and example
Answer: Verify full-depth loss and gradients on every specified step and record the sampled depth. Local diagnostics do not reproduce 20B-token pretraining.
Illustrative example: Even when sampling a shallow exit, confirm full-depth parameters still receive their signal.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Method comparison
How do sampled prefixes differ from training all depths every step?
Show answer and example
Answer: Sampled training gives prefix supervision to selected depths plus the full anchor; all depths cost more per step. Match compute rather than only parameter count.
Illustrative example: One three-layer exit is not equivalent in cost to every exit from one to 12.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Basics
Why support multiple depths in one Telescopic checkpoint?
Show answer and example
Answer: Budgets and inputs can need different depths. Supervise prefixes and full depth so shallow execution is an explicit trained option.
Illustrative example: Choose a shorter prefix rather than train one checkpoint per depth.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Core mechanism
How does MS-GLA avoid leaking later tokens when pooling a block?
Show answer and example
Answer: Each coarse branch masked-averages nonoverlapping blocks and runs GLA. A summary becomes visible only at the block end; causal hold restores token resolution without broadcasting into earlier positions. Input-conditioned weights fuse scales.
Illustrative example: In a four-token block, position two cannot see the average including positions three and four.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Implementation details
When may tokens access scale-four pooled state?
Show answer and example
Answer: At or after the block's final token. Use causal hold and input softmax fusion rather than broadcasting full means to every token.
Illustrative example: Token one cannot use the average of words one through four before word four arrives.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Method comparison
Why is multiscale GLA more than a larger window?
Show answer and example
Answer: Each scale pools at a different nonoverlapping granularity before GLA, changing temporal update resolution. Extending a window still updates at the original rate.
Illustrative example: Updating once per four words differs from updating every word while looking farther back.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Basics
Why give MS-GLA heads different temporal scales?
Show answer and example
Answer: One resolution can bottleneck representations. Some heads preserve token-level variation, others use pooled history, and their outputs combine.
Illustrative example: Phrase detail and trends across phrases can use different update rates.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Core mechanism
How do finitely many recurrent states approximate long fractional-order power-law memory?
Fractional State Space Transition for Long Sequence Modeling
Show answer and example
Answer: One exponential has one decay timescale. Log-spaced modes approximate a power-law tail: fast modes handle recent changes and slow ones keep older influence. Finite capacity and approximation range do not give unlimited exact memory.
Illustrative example: Fast, medium, and slow channels combine into a longer tail than one decay.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Implementation details
What should finite-mode recurrence tests verify besides shape?
Fractional State Space Transition for Long Sequence Modeling
Show answer and example
Answer: Verify equivalence to the finite-mode calculation, long-tail invariants, and state propagation. Operator checks are not reproduction of 1.3B training or model quality.
Illustrative example: Inject an early impulse and compare combined decay over time with the defined approximation.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Method comparison
Why do multiple decay rates approximate a long tail better?
Fractional State Space Transition for Long Sequence Modeling
Show answer and example
Answer: Log-spaced modes span recent and distant timescales, unlike a single exponential. Their finite combination approximates a bounded range, not infinite exact history.
Illustrative example: Fast modes track fluctuations while slow ones retain trends.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Basics
How does power-law memory differ from a recent window?
Fractional State Space Transition for Long Sequence Modeling
Show answer and example
Answer: Old influence fades gradually instead of becoming zero at a fixed boundary. Frac-SSM approximates this with finite exponential modes at several timescales.
Illustrative example: A past event can still matter weakly beyond the latest 100 steps.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Core mechanism
Why edit only the affected subgoal suffix after a failure rather than rewriting the entire plan?
SAGE: Symbolic Action-Gating and Editing for LLM Task Planners
Show answer and example
Answer: The completed prefix reflects actual execution. A full rewrite can repeat actions or wrongly assume the environment reset. The symbolic gate first blocks invalid preconditions and provides a typed cause; the planner then repairs the affected future steps. Keeping the prefix preserves consistency with the current world, not blind commitment to the old plan.
Illustrative example: Example: A cup has been washed, but the cabinet is locked. Revise how to store it rather than washing it again and assuming the cabinet already opened.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Implementation details
Does a zero-token gate establish a low-cost reproduction of the entire LLM planner?
SAGE: Symbolic Action-Gating and Editing for LLM Task Planners
Show answer and example
Answer: No. Only the symbolic gate uses no language-model calls. Plan generation and suffix editing still have their own costs. The local run executes the gate and a reference suffix editor, not AI2-THOR. Three-seed fixture success is neither embodied task success nor end-to-end inference cost.
Illustrative example: Example: A Boolean check can refuse to open a cabinet without producing the new plan. Counting only the check misses recovery's model calls.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Method comparison
What do typed failure causes add over a free-text message saying an action failed?
SAGE: Symbolic Action-Gating and Editing for LLM Task Planners
Show answer and example
Answer: Typed causes distinguish missing resources, unmet preconditions, and affected subgoals, allowing a local editor to revise the relevant suffix. Generic text can cause a full rewrite or repetition of completed actions. Structured feedback still depends on accurate symbolic state and does not replace real environment feedback.
Illustrative example: Example: A locked cabinet calls for a key or another location; a missing object calls for renewed perception. Those are different recoveries.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Basics
Why check preconditions before executing an action in a symbolic action gate?
SAGE: Symbolic Action-Gating and Editing for LLM Task Planners
Show answer and example
Answer: Preconditions tell whether the current world permits an action. Checking first blocks known illegal or unsafe actions and gives the planner a typed reason. The gate covers only modeled rules. Passing it does not prove real-world safety when rules or state are incomplete.
Illustrative example: Example: Grasping requires an empty hand and reachable object. The gate blocks a grasp with a full hand before execution rather than explaining a collision afterward.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why separate slow search planning from fast record decisions instead of reasoning at length about every record?
Mnemon: Raw Records, Fast Judgments, Slow Thoughts
Show answer and example
Answer: Planning chooses where to seek evidence; per-record eligibility fits a bounded fast decision. Separation spends reasoning budget on search direction rather than repeatedly explaining candidates. Dated originals remain, while the answerer sees a small budgeted view. Fast decisions do not guarantee complete evidence or correct time eligibility.
Illustrative example: Example: Decide to search a release meeting, quickly screen dozens of records, then send a few originals to the answerer instead of analyzing every record at length.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
Does the local bounded-view test cover Jev semantic judgments and the original answer model?
Mnemon: Raw Records, Fast Judgments, Slow Thoughts
Show answer and example
Answer: No. It runs raw-record retrieval and budgeted view construction without a Jev checkpoint or reproducing LoCoMo scores. Record order, dates, and budget are independently checkable. Real models are needed to test semantic necessity and whether the answerer uses evidence correctly.
Illustrative example: Example: Fitting three specified records verifies the view interface, not that the judge selects them from natural language.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
How do dated raw records compare with repeatedly rewriting one long-term summary for traceability?
Mnemon: Raw Records, Fast Judgments, Slow Thoughts
Show answer and example
Answer: Raw records retain exact wording, time, and details for different questions, but cost storage and query-time reading. A rolling summary is compact yet can erase provenance or rare facts. Mnemon builds a budgeted view from originals for this query, not a permanent lossless replacement.
Illustrative example: Example: A summary says the customer prefers A, while the dated email says only this quarter. Check the original before answering about next quarter.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Why does Mnemon keep dated originals but give the answering model only a small view?
Mnemon: Raw Records, Fast Judgments, Slow Thoughts
Show answer and example
Answer: Originals retain checkable details and dates. A small view gives relevant evidence within budget instead of replaying all history. Slow planning proposes searches and fast Jev decisions select records. Missing facts may require further lookup; the view is not a lossless summary.
Illustrative example: Example: Pick a few dated rescheduling events from years of records rather than resend all chats to answer when the latest change happened.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Core mechanism
How can test-time adaptation occur with frozen task-model weights?
Harness Learning Enables Generalizable Test-Time Adaptation
Show answer and example
Answer: Adaptation changes executable components outside the model, such as verification, interpreters, retrieval, and retry protocols. Execution feedback changes that scaffolding and therefore later behavior of the same model. This is not fine-tuning and cannot route hidden answers through the interface to a proposer. Local classification-policy updates are not the paper's full LLM adaptation chain.
Illustrative example: Keep the model fixed, but add shell-output truncation and error localization after a failure. The next attempt can process the output better without any neural-weight change.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Implementation details
Why do external callbacks not establish integration with a general agent runner or Evolve?
Harness Learning Enables Generalizable Test-Time Adaptation
Show answer and example
Answer: A trusted callback is an execution interface. A general system must safely run arbitrary harnesses and isolate validation. That full chain is missing. Adaptation inputs allow only scores and failure categories, not gold data or hidden verifier fields. Mini-suite success is not open-ended agent capability.
Illustrative example: A callback that returns a preset high score can test loop control, but without real candidate execution it supplies no promotion evidence.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Method comparison
How does within-group relative-reward training of the revision policy differ from fixed hand-written rules?
Harness Learning Enables Generalizable Test-Time Adaptation
Show answer and example
Answer: Locally, subtract the group's mean execution score from each candidate score to form advantages and apply REINFORCE to revision logits. Feedback can change proposal preferences; fixed rules learn no parameters. The small classification proposer is not the paper's general harness learner, and a formula alone does not establish safe arbitrary component changes.
Illustrative example: A revision that performs better within its group gets positive advantage and becomes more likely, instead of being predetermined by a rule author.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Basics
Why does Harness Learning treat tool protocols, memory, and planning as learnable objects?
Harness Learning Enables Generalizable Test-Time Adaptation
Show answer and example
Answer: With the same weights, the external workflow still controls what the model sees, when it calls tools, and how it recovers. Harness Learning uses feedback to revise executable organization rather than attributing every weakness to the base model. Workflow adaptation is a distinct scope and does not imply new capability learned in model parameters.
Illustrative example: Add testing before every submission without changing the base model. Outcomes may change because execution was organized differently, not because weights were trained.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
Why solve a discounted state-graph fixed point before assigning action credit by differences in log potential?
GraphHCA: Closed-Form Hindsight Credit Assignment for Long-Horizon LLM Agents
Show answer and example
Answer: Propagate terminal successes and failures as boundary conditions over observed transitions to obtain discounted success potential. Differences in adjacent log potentials turn long-range evidence into local progress, then standardize across actions from a shared state. Complete rollouts support training attribution; do not expose hindsight success boundaries as observations to evaluated agents.
Illustrative example: Two paths through one intermediate state can share downstream success evidence. Compare whether actions leaving that state approach success or lead toward a dead end.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
Why do local Transition and success inputs stay with the estimator rather than the actor?
GraphHCA: Closed-Form Hindsight Credit Assignment for Long-Horizon LLM Agents
Show answer and example
Answer: Terminal labels estimate training-side potential. The actor reads only the public state then available, not future success boundaries. No language model was trained or integrated into Evolve. Fixed-point convergence and credit rankings are mechanism evidence; full integration needs real rollouts, updates, and isolated evaluation.
Illustrative example: Labeling a terminal state after training interaction can help compute credit. Telling a test agent which successor leads to success leaks the result.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
How do differences in graph potential differ from broadcasting one advantage to the whole trajectory?
GraphHCA: Closed-Form Hindsight Credit Assignment for Long-Horizon LLM Agents
Show answer and example
Answer: Trajectory advantage does not distinguish steps moving toward promising states. Potential differences highlight improved or worsened state value and are standardized among actions sharing a start. They depend on empirical transition probabilities and ε, not independent causal contribution or guaranteed loop penalties.
Illustrative example: In trajectories with the same terminal result, moving to high potential and returning to low potential can get different local credits instead of one shared value.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why does GraphHCA give long-horizon actions a structured training signal before terminal success?
GraphHCA: Closed-Form Hindsight Credit Assignment for Long-Horizon LLM Agents
Show answer and example
Answer: Sparse terminal feedback can obscure early exploration and key transitions behind one trajectory reward. GraphHCA organizes observed transitions in a graph and propagates success to connected states to estimate action progress. The graph covers observed data only, its potential is not ground truth, and deployment actors cannot see training success labels in advance.
Illustrative example: If several routes reach success after opening a key door, the graph connects that experience. It cannot assume success behind an unseen door.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
Why shouldn't retrieving duplicate evidence repeatedly earn the same process reward?
Dr.Credit: Rubric-Grounded Process Credit Assignment for Deep Research Agents
Show answer and example
Answer: Credit should measure new support relative to accepted evidence, not tool calls. Track historical rubric coverage and score new or partial support to discourage reward farming. Retain outcome advantage so local novelty remains aligned with research goals, and require verifiable support relationships.
Illustrative example: Retrieving the same date twice earns no full second reward without a new constraint. Finding previously missing affiliation evidence adds support.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
What can local fixture rubrics and support scores validate?
Dr.Credit: Rubric-Grounded Process Credit Assignment for Deep Research Agents
Show answer and example
Answer: They validate history updates, no repeat credit for duplicates, and direction of incremental credit. They do not test whether a semantic judge correctly recognizes support. The policy does not read gold answers, and a three-seed mini-suite is not Deep Research capability. Stronger claims need real sources, trustworthy matching, and actual training.
Illustrative example: A fixture stating that evidence A supports criterion one can test that the second retrieval adds zero credit. It does not test whether a model recognizes that support on a webpage.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
How does incremental rubric-based process feedback differ in timing from grading only the final answer?
Dr.Credit: Rubric-Grounded Process Credit Assignment for Deep Research Agents
Show answer and example
Answer: Terminal scoring judges the deliverable but cannot easily isolate useful retrievals. Incremental support checks when individual requirements gain evidence and links feedback to those actions. Supported evidence can still be misused in the final answer; neither signal replaces the other, and evaluated actors must not see the training rubric.
Illustrative example: A retrieval can find the correct date and complete a process requirement while the final answer copies the year incorrectly. Terminal scoring should still mark that error.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why does Dr.Credit ask whether each deep-research retrieval adds new evidence for the answer?
Dr.Credit: Rubric-Grounded Process Credit Assignment for Deep Research Agents
Show answer and example
Answer: An agent can search repeatedly and collect near-duplicates without completing the facts needed. Dr.Credit links process feedback to rubric support so retrieval actually advances the answer. Rewards come from training-side judgments of support, not document counts; judge and rubric reliability still limit credit quality.
Illustrative example: Three reposts of a known price do not establish warranty terms. Finding the original warranty can add a missing answer requirement.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Core mechanism
Why bootstrap by task clusters for an automated-region error certificate instead of treating all traces as independent?
Certified Selective Automation of LLM Agent Evaluation
Show answer and example
Answer: Traces from the same task share difficulty and structure, so independent resampling understates correlation and uncertainty. Cluster sampling builds an error upper bound; automate only below budget, with human or external review otherwise. Low average error does not certify arbitrary thresholds, and the certificate depends on calibration distribution and selection.
Illustrative example: A hundred similar traces from one task are not a hundred independent successes. Resample them as one task cluster.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Implementation details
Can the local bootstrap certificate be reported as coverage on the paper's corpus?
Certified Selective Automation of LLM Agent Evaluation
Show answer and example
Answer: No. The mini-suite tests clustered resampling and the upper-bound takeover rule. Fix traceable task identities, automated regions, and error labels. The confidence bound rather than point error must satisfy budget.
Illustrative example: Low sample error with a high bound does not establish that the error budget is met.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Method comparison
What does an error upper confidence bound control beyond empirical error in the automated region?
Certified Selective Automation of LLM Agent Evaluation
Show answer and example
Answer: Empirical error summarizes observed samples, while the upper bound reflects uncertainty from finite evidence. Automate only below budget; even zero observed errors can be insufficient. Certification concerns risk in the selected region, not guaranteed correctness on every item.
Illustrative example: No observed errors in a small region does not imply a low enough upper bound or justify expanding coverage.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Basics
Why can selective automated evaluation defer some cases to people?
Certified Selective Automation of LLM Agent Evaluation
Show answer and example
Answer: The aim is to automate reliable regions within a certified error budget, not apply machine judgments to every trace. If the upper confidence bound fails the budget, retain human or other evaluation. Report coverage together with risk.
Illustrative example: Automate clear cases while routing ambiguous ones to reviewers; accuracy on automated cases alone is incomplete.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
If the training teacher sees full history, why can the next student turn see only bounded memory?
Show answer and example
Answer: Privileged history supplies training signals, while the deployed student still sees the task, memory, and latest observation. This forces it to write necessary state into limited space. Providing full history at evaluation bypasses continuous management. Memory updates accompany actions and must serve the next decision.
Illustrative example: Example: A teacher knows a port number mentioned earlier. The student must preserve it in small memory rather than receive the whole earlier conversation on test.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
How do you check that CCM does not bypass bounded context with unrecorded history?
Show answer and example
Answer: Inspect that each policy request contains only the original task, current memory, and latest observation. Prior tool results are visible only if written into memory, which has a hard cap. The local run tests these contracts without TerminalBench or WebShop checkpoints; field isolation is not task capability.
Illustrative example: Example: Hiding a full previous transcript in an extra metadata field violates the contract even if the main prompt is shorter.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
How does per-turn bounded state differ from retrieving a few old records just before answering?
Show answer and example
Answer: CCM updates memory during each action, so retention now determines next-turn visibility. Answer-time retrieval makes the main selection later using an external index. The bottlenecks differ and can complement each other. Bounded-state omissions cannot rely on hidden history, and retrieval must count indexing and reads.
Illustrative example: Example: A terminal changes directories. CCM should immediately keep the path; later retrieving an old command alone may miss the directory change.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
What kind of context growth is CCM trying to avoid across many turns?
Show answer and example
Answer: Resending every observation, action, and thought grows cumulative input and can bury current state. CCM lets the agent maintain bounded memory each turn and keep acting. It studies how to compress the past into decision-sufficient state rather than retain complete history indefinitely.
Illustrative example: Example: At step ninety, current files, verified results, and pending work may suffice without every terminal output from the previous 89 steps.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Core mechanism
Why does FLVM place measurement confounders in a restricted baseline and correct feedback heads through routed latents?
Mend the Measurement Gap: Latent User Preference Modeling for Short-Form Video Recommendation
Show answer and example
Answer: Watch time reflects length and user habits, not pure preference. The baseline explains measurement conditions; separate consumption, active engagement, and polarity factors reach only relevant heads. Stop-gradient prevents the residual objective from rewriting its reference. These constraints do not fully identify causal latents.
Illustrative example: Similar watched seconds can come from completing a short clip or abandoning a long one; length opportunity and active liking should not become one score.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Implementation details
Why distinguish FLVM feedback masks from fixed latent routing masks?
Mend the Measurement Gap: Latent User Preference Modeling for Short-Form Video Recommendation
Show answer and example
Answer: Observation masks indicate whether sample labels exist; routing masks restrict which factors heads read, on different axes. Missing labels are not zero, baseline addition stops the specified gradients, and ranking uses the defined combination of consumption and polarity rather than rewarding activity as positive.
Illustrative example: An unobserved dislike is not a like. Even when observed, a dislike head cannot read unrelated factors to bypass routing.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Method comparison
What shortcut do FLVM's constrained baseline and stop-gradient restrict compared with a shared multitask tower?
Mend the Measurement Gap: Latent User Preference Modeling for Short-Form Video Recommendation
Show answer and example
Answer: Duration and time first explain the measurement baseline. Full user-item information contributes via latents and fixed semantic routing, while baseline gradients train separately. This limits unrestricted mixing but does not remove all confounding; local factor separation was not stable.
Illustrative example: Let length explain greater opportunity to accumulate seconds, rather than using long as a direct synonym for liked.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Basics
Why separate consumption, active engagement, and direction in FLVM?
Mend the Measurement Gap: Latent User Preference Modeling for Short-Form Video Recommendation
Show answer and example
Answer: Watching, acting, and liking are different, and negative feedback can be highly active. Restricted heads use relevant factors rather than counting all engagement positively. Latents do not automatically identify psychological concepts and require semantic checks.
Illustrative example: Finishing a controversial video and disliking it gives high consumption and engagement but low positive direction.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why compress and cache individual behaviors in KuaFu rather than compressing a user's entire history into one vector first?
KuaFu: Compressing Long User Behavior into Understanding at Billion Scale
Show answer and example
Answer: Individual behavior encodings can be reused across users or profiling tasks, and a new event needs only its cache entry. Compressing the whole history binds shared content to user context and can force recomputation on every change. The decoder reads compressed events chronologically and can compose them. Compression loses information, and cache efficiency must not hide the negative local diagnostic.
Illustrative example: Illustration: Several users can reuse the encoding of the same video behavior. When one watches another video, append its cached representation and decode in order rather than re-encoding all old text.
Sections in the Chinese source: 背景与主要改动, 本地复现. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
Why should KuaFu report skipped_zero_advantage when every reward in the RL smoke test is equal?
KuaFu: Compressing Long User Behavior into Understanding at Billion Scale
Show answer and example
Answer: Identical group rewards yield no relative advantage and no usable policy-gradient signal. Invoking the optimizer does not mean RL succeeded. The source preserves the skipped update and verifies QA compression training separately from RL. A GPU forward pass does not prove successful reward optimization.
Illustrative example: Illustration: Four answers each score one, so their normalized advantages are zero. The model cannot prefer any answer from that signal; report no update instead of calling the four high rewards learning.
Sections in the Chinese source: 本地复现. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
Why do KuaFu's input ablation and independently trained raw-text baseline have different evidentiary strength?
KuaFu: Compressing Long User Behavior into Understanding at Billion Scale
Show answer and example
Answer: Switching a decoder trained on compressed inputs to raw text confounds the comparison with adaptation to the input distribution. Training the same model separately under the raw-text budget better compares the training paths. The source remained negative after the independent control, and the comparison was not formal equal-FLOPs evaluation, so do not report compression efficiency alone.
Illustrative example: Illustration: A decoder trained only on compressed codes might struggle with truncated text because it is unfamiliar. Training the raw-text branch independently reduces that confound.
Sections in the Chinese source: 背景与主要改动, 本地复现. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why does KuaFu compress both memory token count and channel dimension?
KuaFu: Compressing Long User Behavior into Understanding at Billion Scale
Show answer and example
Answer: A behavior first has a high-dimensional memory representation, then is projected into fewer, lower-dimensional cached vectors. This saves sequence positions and storage bandwidth separately, and the decoder composes entries in sequence order. Two-axis compression can lose information; reconstructing some answers is not evidence of preserving every temporal or causal relationship.
Illustrative example: Illustration: Compressing a 24-word event into two low-dimensional vectors saves more storage than merely halving the word count, but names or times may disappear. Measure that with task metrics, not compression rate alone.
Sections in the Chinese source: 背景与主要改动, 本地复现. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Core mechanism
Why impose different gold boundaries on DCE and SRCL although both update one model?
Recursive Self-Improvement via On-Policy Distillation for Reasoning
Show answer and example
Answer: DCE uses current privileged teacher to score actual prefixes, then refreshes both roles after updates. SRCL rewrites its own answer without gold and validates afterward. Giving gold to rewriter changes self-revision into copying. No second teacher is independently trained.
Illustrative example: Score a student attempt using a reference solution, then let it shorten its own response with correctness checked only afterward.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Implementation details
What if no shorter revision qualifies in Recursive OPSD?
Recursive Self-Improvement via On-Policy Distillation for Reasoning
Show answer and example
Answer: Run DCE only; don't force failed or longer rewrites into SFT. Identify prompt-role differences in frozen teacher controls; the local assistant-side arm isn't the original user-side OPSD baseline.
Illustrative example: A correct but longer answer does not pass SRCL and must stay excluded.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Method comparison
How does SRCL rewriting differ from showing a teacher gold to rewrite?
Recursive Self-Improvement via On-Policy Distillation for Reasoning
Show answer and example
Answer: Rewrite sees only problem and its own response. Gold is for acceptance. Require shorter, self-contained, naturally ended and correct outputs to avoid mistaking privileged copying for autonomous revision.
Illustrative example: Do not include the reference solution in the shortening prompt.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Basics
How does Recursive OPSD DCE keep teacher aligned to student updates?
Recursive Self-Improvement via On-Policy Distillation for Reasoning
Show answer and example
Answer: Both use current shared weights, teacher additionally sees the reference solution and is detached. Updated weights become both views next round, not a separately trained or initial teacher.
Illustrative example: A newly learned correction is available to the next privileged teacher.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Core mechanism
Why swap both word identity and document order in conflict tests?
Words Speak Louder Than Order: A Behavioral Evaluation of Gemma 4
Show answer and example
Answer: Order alone confounds lexical preference, source role, and position. Four orthogonal crossings estimate effects, with a shared filtered set and unfiltered sensitivity analysis. Conclusions are for one model on short English choices.
Illustrative example: A favored word can win in either position; swap which source provides it to isolate source and order.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Implementation details
Can failed or degenerate BCa intervals silently fall back to equivalence?
Words Speak Louder Than Order: A Behavioral Evaluation of Gemma 4
Show answer and example
Answer: No. Keep failed/degenerate and equivalent=null, not a hidden percentile interval. Ten conditions share the union of exclusions, and unfiltered sensitivity remains.
Illustrative example: If bootstrap cannot form a valid interval, report undecided rather than a pass.
Sections in the Chinese source: 核心公式, 实现与数据协议. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Method comparison
Why distinguish order null contrast O from positional effect S?
Words Speak Louder Than Order: A Behavioral Evaluation of Gemma 4
Show answer and example
Answer: They use different sign combinations. S measures position and negative S favors first; O is a null control. Both are half effects, so double for full range.
Illustrative example: Changing the signs on e1–e4 yields a different contrast.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Basics
Why use a multiway crossing for two-document conflicts?
Words Speak Louder Than Order: A Behavioral Evaluation of Gemma 4
Show answer and example
Answer: Word identity, source, and position can all influence choice. Four conditions separate them instead of mistaking lexical or ordering preference for trust.
Illustrative example: Preferring word B is different from preferring whichever word the new source supplies.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why condition flow on normalized item space instead of generating SID tokens?
Show answer and example
Answer: Continuous triggers avoid code quantization and use ANN directly. Coarse anchors condition geodesic noise-to-item flow; tangent projection preserves geometry. Cache earlier history layers and denoise final-layer tokens. Multistep cost remains and local low-budget outcomes are negative.
Illustrative example: Generate camera/lens directions around a photography anchor rather than decoding integer IDs.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Does local X-Rec have a GPU throughput receipt, and what runs?
Show answer and example
Answer: No. CPU runs scaled anchors, geodesic RFM, final-layer KV reuse, and spherical Euler. No industrial ANN, streaming, or online A/B; CPU timings are not production throughput.
Illustrative example: Caching history verifies skipped computation without proving A100 gains.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
How does spherical flow differ from Euclidean interpolation?
Show answer and example
Answer: Geodesic paths and tangent projection preserve normalization. Straight lines can leave the sphere. Reusing earlier history layers is a separate efficiency design.
Illustrative example: The midpoint of two unit vectors is generally not unit length.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Why generate X-Rec triggers on the item embedding sphere?
Show answer and example
Answer: Normalized continuous vectors query multiple regions directly without SID quantization or token-by-token decoding. Anchors condition flow toward valid item neighborhoods.
Illustrative example: Generate different interest directions and retrieve nearby catalog items.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Core mechanism
Why connect the Research Agent and Model Agent through proposals instead of sharing full experiment logs?
Advancing Model Research in AgentX: Long-Horizon Autonomy for Industrial Recommender Systems
Show answer and example
Answer: Proposals specify the question, constraints, and approved direction. The model implementer runs iterative experiments under fixed business inputs and prediction tasks, preventing mutual context overload. Return interim bests, negative results, and unresolved questions for the next research decision. The interface carries evidence, not just one appealing final score.
Illustrative example: Researchers can propose a long-history compression hypothesis. The model agent reports the best version and when it fails, allowing the next hypothesis without showing every compilation log.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Implementation details
When are candidate outcomes revealed in historical replay, and how does local data avoid future leakage?
Advancing Model Research in AgentX: Long-Horizon Autonomy for Industrial Recommender Systems
Show answer and example
Answer: A child can be selected only after its parent, and the candidate's result is revealed only after selection. MovieLens holds out the last two chronological interactions for validation and test; training statistics cannot read future events, and champions use validation only. A rule-based selector and logistic-regression replay are not the paper's LLM research agent or an online A/B replication.
Illustrative example: Even if the logs contain a high score from another branch, the selector cannot read it before deciding to try that branch. Otherwise it has a future-metric oracle.
Sections in the Chinese source: 本地复现, 可执行路径与防泄漏边界. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Method comparison
Why does improvement over a parent differ from setting a new best over the entire ancestral path?
Advancing Model Research in AgentX: Long-Horizon Autonomy for Industrial Recommender Systems
Show answer and example
Answer: If a parent regressed, its child can recover some performance without beating an older champion. Track both the parent delta and comparable ancestral delta to avoid presenting recovery as a breakthrough. Diagnose actions primarily reduce uncertainty and should not be ranked by the same immediate AUC objective as improvement actions.
Illustrative example: If an ancestor has AUC .80, parent .75, and child .78, the child gains .03 over its parent but is still .02 below the path best. Report both.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Basics
Why does AgentX-Model need to connect each research choice to actual experiments?
Advancing Model Research in AgentX: Long-Horizon Autonomy for Industrial Recommender Systems
Show answer and example
Answer: Automated research must choose what to try and obtain new evidence through implementation and evaluation, not just generate proposals. Research and model execution roles exchange proposals so search history reflects attempted candidates. Historical replay only reenacts revelation and selection of existing outcomes; it does not rerun training locally.
Illustrative example: After a proposal to change the loss, the implementer returns results only after training. A replay should likewise reveal a score only after selecting the candidate, without peeking at future outcomes.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why share user context but forbid cross-stage token attention in OneTrans-V2?
Show answer and example
Answer: Three stages can reuse costly behavior encoding while retaining distinct inputs and objectives. Each reads only prior behavior, avoiding downstream or future leakage. Fine-to-pre-ranking knowledge travels through controlled distillation, not unrestricted reads. Local work lacks independent three-stage controls.
Illustrative example: Illustration: All stages share observed browsing, but retrieval cannot see later clicks or fine-rank labels. Preranking learns from a stopped-gradient teacher.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why should OneTrans-V2 decision offsets directly change only decision logits?
Show answer and example
Answer: The local test offsets two observable proxy decisions. Given a decision, it should not directly rewrite SID distributions or pre-/fine-rank outputs. This tests interface independence, not unobserved purchase value or ad supply. Proxy thresholds are not CTR/CVR.
Illustrative example: Illustration: Raising the discovery offset can change decision likelihood, not add a constant to all fine-rank scores.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
Why aren't independent stage attention and fine-to-pre-rank distillation contradictory?
Show answer and example
Answer: Forward isolation controls serving inputs; training distillation transfers stopped-gradient soft supervision into parameters. It need not expose the fine-ranker online. Runnable stages without an independent three-stage baseline do not establish cascade gains.
Illustrative example: Illustration: Teach preranking to mimic fine ranking offline. Running fine ranking first on every request defeats ordering and cost.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why keep three stages' own inputs and targets while sharing OneTrans-V2 user context?
Show answer and example
Answer: Retrieval, preranking, and fine ranking have different candidate counts and features. Shared causal history saves encoding, while stage-specific tokens preserve constraints and controlled losses transfer knowledge. A stage cannot read another's future labels.
Illustrative example: Illustration: Retrieval sees codes while fine ranking sees item details; shared history does not expose posteriors to retrieval.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Core mechanism
Why predict a joint like-and-click label when both marginal rates are already predicted?
Learned Cross-Task Relationships in Multi-Task Models
Show answer and example
Answer: Marginals do not determine whether feedback occurs on the same examples. A joint head models dependence for shared representation supervision and downstream decisions; validate the benefits separately. Check feasibility when reconstructing the distribution rather than clipping negative probabilities and claiming calibration.
Illustrative example: Both rates can be 20% with complete overlap or complete separation. Marginals alone cannot tell how to combine objectives.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Implementation details
Why not silently clip negative probabilities when reconstructing four joint cells from pX, pY, and pXY?
Learned Cross-Task Relationships in Multi-Task Models
Show answer and example
Answer: The cells must be nonnegative and sum to one; pXY is also bounded by the marginals. Negatives expose inconsistent predictions. Preserve an infeasible marker instead of clipping and claiming calibration; observed labels belong in loss and evaluation, not pre-impression inputs.
Illustrative example: pX=0.2 and pXY=0.4 imply negative click-only probability, which should be reported as inconsistent.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Method comparison
Why compare true cross-label auxiliary supervision with shuffled labels rather than only adding or removing parameters?
Learned Cross-Task Relationships in Multi-Task Models
Show answer and example
Answer: An extra head can change the backbone through more gradients or regularization. Shuffling keeps the training form while destroying real task relations, helping isolate the relation signal. A sparse small slice is still only diagnostic and does not support stable capability claims.
Illustrative example: If real click-and-long-watch labels help but equal-sized shuffled labels do not, that supports useful joint information; similar results may reflect auxiliary training alone.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Basics
Why can multitask models with the same marginals predict different joint behavior?
Learned Cross-Task Relationships in Multi-Task Models
Show answer and example
Answer: Marginals give frequencies, not co-occurrence. Covariance is joint probability minus the product of marginals, adding dependence information to shared representations and decisions. The task pair needs enough nondegenerate combinations; mutually exclusive or constant labels do not support the same interpretation.
Illustrative example: Click and long-watch can each have probability one half while always coinciding or never coinciding.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Core mechanism
Why recompute PACT ratios after actor update, and may targets above one be clipped?
PACT: From Credit Assignment to Critic Alignment
Show answer and example
Answer: New/old token ratios correct old returns toward the updated actor's critic target. The product can exceed one; clipping changes the objective. Use stable softplus and the prescribed update order.
Illustrative example: A more likely successful action after update should change critic weighting instead of fitting only old sampling.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Implementation details
Why not send ρR above one to ordinary probability BCE?
PACT: From Credit Assignment to Critic Alignment
Show answer and example
Answer: Use the specified softplus(z)−Yz form and filter out-of-range ratios. Standard BCE may reject or alter targets outside [0,1]. Sparse nonzero local outcomes aren't reliable gains.
Illustrative example: Reward one times a ratio slightly above one naturally exceeds one; do not clamp it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Method comparison
Why update actor before critic in PACT instead of fitting the old policy?
PACT: From Credit Assignment to Critic Alignment
Show answer and example
Answer: The critic should follow the updated policy. Correct observed returns with a current-token new/old ratio, not an exact full-future continuation product.
Illustrative example: Once actor actions change, replaying uncorrected old returns leaves the critic behind.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Basics
Why tie PACT token credit to prefix value?
PACT: From Credit Assignment to Critic Alignment
Show answer and example
Answer: One terminal reward is coarse. Prefix-conditioned expected return contextualizes it; R−V approximates advantage. Exact credit needs theoretical assumptions; a learned critic is an approximation.
Illustrative example: The same failure can yield different advantages at an easy versus difficult prefix.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Core mechanism
Can a new Light Heads objective improve the backbone after a stop-gradient on shared features?
Lightweight Ranking Heads: Accelerating Multi-Task Experimentation in Production Recommender Systems
Show answer and example
Answer: Not through that loss path: it trains the shallow head but cannot update the shared backbone, reducing interference with existing tasks. Window resets make head training ages comparable but may sacrifice fit on sparse objectives. Gradient isolation and resets are separate; improved head AUC does not establish an improved backbone.
Illustrative example: Illustration: Add a feedback head without harming a mature click tower; resetting weekly supports fair upstream comparisons but may leave a rare objective undertrained.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Implementation details
What should be checked separately for Light Head gradient isolation and reset?
Lightweight Ranking Heads: Accelerating Multi-Task Experimentation in Production Recommender Systems
Show answer and example
Answer: With only the new-task loss, shared θ must receive zero gradient while light-head ψ receives nonzero gradient. At a window boundary, reinitialize only ψ and preserve existing θ and main head φ as specified. Resetting does not imply isolation; do not erase backbone optimizer state accidentally.
Illustrative example: Illustration: A changed shared parameter after the new-task update violates isolation. Resetting the main click head too would destroy the baseline.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Method comparison
Why trade off training-age comparability against sparse fitting when resetting a Light Head?
Lightweight Ranking Heads: Accelerating Multi-Task Experimentation in Production Recommender Systems
Show answer and example
Answer: Resetting each window equalizes training age across upstream variants. Keeping the head preserves sparse label experience and may fit better. Short-window resets clearly regressed in the local source, so the right choice depends on the experimental goal and available data.
Illustrative example: Illustration: A head trained three weeks versus a head trained one day confounds an upstream comparison. Reset both for a fairer comparison, but rare labels may not be enough to relearn.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Basics
Why do Light Heads introduce a shallow head instead of changing the shared backbone immediately?
Lightweight Ranking Heads: Accelerating Multi-Task Experimentation in Production Recommender Systems
Show answer and example
Answer: A production system can try a new objective quickly while protecting existing tasks and interfaces. The head reads shared features and updates only itself, making a cheap probe. It can exploit but not improve the backbone through this loss, and cannot recover information absent from those features.
Illustrative example: Illustration: Probe sparse satisfaction on current user embeddings without changing the click network. Adding a head cannot conjure details the embeddings never captured.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why use flat codes and forbid candidates from reading each other?
UNIQUE: A Unified Retrieval and Ranking System for Large-Scale Feed Recommendation
Show answer and example
Answer: Flat allocation avoids error accumulation and uses feedback, frequency correction, and EMA. Shared prefixes save user encoding; item/code isolation keeps scoring independent. Joint objectives connect retrieval and ranking; monitor use and collisions.
Illustrative example: Retrieve one-level code buckets and score each candidate against the same history, not its neighbors.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Do straight-through quantization and balanced code usage establish retrieval quality?
UNIQUE: A Unified Retrieval and Ranking System for Large-Scale Feed Recommendation
Show answer and example
Answer: No. Straight-through preserves gradients and correction helps usage, not relevance. Three small seeds do not rebuild industrial indexes, P99, or online gains.
Illustrative example: Equally sized buckets can contain uniformly irrelevant mixtures.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
How does shared-prefix-only candidate attention differ from setwise attention?
UNIQUE: A Unified Retrieval and Ranking System for Large-Scale Feed Recommendation
Show answer and example
Answer: Candidate isolation reuses user encoding and avoids peers changing scores. Setwise attention depends on the candidate set. Unified retrieval and ranking do not require item-to-item reads.
Illustrative example: Adding a candidate should not alter an existing candidate through peer attention.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Why use a flat UNIQUE code instead of residual levels?
UNIQUE: A Unified Retrieval and Ranking System for Large-Scale Feed Recommendation
Show answer and example
Answer: It avoids stagewise quantization error, using feedback towers, frequency correction, and EMA for one level alongside retrieval and ranking losses. Flat codes need not uniquely identify items.
Illustrative example: Retrieve one semantic bucket, then rank its items.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why generate Music Rationales offline and separately test nomination and showing rationales?
Explainable Recommendations at Scale: LLM Rationales for YouTube Music Artist Discovery
Show answer and example
Answer: Offline generation uses full history and expensive reasoning, then normalizes entities and filters rationales for cached serving. Nomination changes candidates; showing reasons can change response to the same candidates. Separate arms identify which path contributed. Fluency does not establish trustworthy reasons.
Illustrative example: Illustration: Show the same unfamiliar jazz artist with or without a reference to the user's piano-trio taste to isolate the reason's effect.
Sections in the Chinese source: 背景与主要改动, 核心公式与机制. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
Why retain rejection and cache-miss states in Music Rationales?
Explainable Recommendations at Scale: LLM Rationales for YouTube Music Artist Discovery
Show answer and example
Answer: Normalize artists to a real catalog, reject unsupported explanations, and fall back to the original recommender on a cache miss rather than inventing text. Evaluate nominations separately from final fallback lists. A single-seed diagnostic without explanation CTR cannot establish user benefit.
Illustrative example: Illustration: Fall back to CF on a nonexistent artist rather than force a mistaken name match; fallback availability is not LLM nomination success.
Sections in the Chinese source: 背景与主要改动, 核心公式与机制. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
What questions do explanation-only with fixed CF ordering and LLM-first nomination answer?
Explainable Recommendations at Scale: LLM Rationales for YouTube Music Artist Discovery
Show answer and example
Answer: Nomination changes coverage and position and can affect offline hits. Adding explanations to the same list keeps offline Hit identical; only rationale coverage can be checked offline. Actual behavioral response needs a user experiment. Separate arms prevent misattribution.
Illustrative example: Illustration: Adding reasons for the same five artists cannot change static Hit@5 but might change click behavior under real exposure.
Sections in the Chinese source: 背景与主要改动, 核心公式与机制. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why include unplayed artists and personalized reasons in a Music Rationales discovery profile?
Explainable Recommendations at Scale: LLM Rationales for YouTube Music Artist Discovery
Show answer and example
Answer: A reason connects unfamiliar candidates with known interests. Generate nomination and explanation asynchronously, then verify entities and trustworthiness. It supports discovery, not guaranteed taste fit, and must not invent history to persuade.
Illustrative example: Illustration: Explain how an artist relates to piano trios the user plays, but do not claim a prior listen absent from history.
Sections in the Chinese source: 背景与主要改动, 核心公式与机制. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Core mechanism
Why compress recent, medium, and distant history at different strides and let candidates choose MuSeR interests?
MuSeR: Scalable Long-sequence Recommendation with Multi-interest Modeling
Show answer and example
Answer: Detailed recent events preserve immediate intent; larger distant strides control cost. Learned queries extract multiple interests, and candidate similarity selects a mixture. Compression determines how much history to read; candidate conditioning determines what to use. Pooling can still lose rare old events.
Illustrative example: Illustration: Summarize long-running photography and retain each recent camping view. A camera and tent activate different interests.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Implementation details
Why report absolute scores and component results alongside large relative local MuSeR Recall gains?
MuSeR: Scalable Long-sequence Recommendation with Multi-interest Modeling
Show answer and example
Answer: A low baseline makes small absolute changes look large in percentage terms. The hierarchical compression ablation was small, so do not credit it with the joint gain. State user sampling and full-catalog scope; label proxies do not establish original LLM summary gains.
Illustrative example: Illustration: Recall from 1% to about 2.6% looks large relatively, but most targets still miss the top ten.
Sections in the Chinese source: 本地复现. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Method comparison
What conditioning does MuSeR preserve by selecting interests per candidate instead of averaging?
MuSeR: Scalable Long-sequence Recommendation with Multi-interest Modeling
Show answer and example
Answer: A common mean gives every item the same summary. Candidate softmax reads different interests according to similarity, requiring candidate-specific weights rather than wholly shared cache state. Local label fusion is not the full ERNIE/BGE multimodal pipeline.
Illustrative example: Illustration: Cameras emphasize photography and tents emphasize camping; averaging weakens each relevant signal.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Basics
Why use an orthogonality regularizer for MuSeR's learned interest queries?
MuSeR: Scalable Long-sequence Recommendation with Multi-interest Modeling
Show answer and example
Answer: Several queries may all read the same popular pattern. Penalizing similar interest directions encourages specialization for candidate selection. Geometry does not ensure semantic complementarity and can separate genuinely related interests. Use ablations and task metrics.
Illustrative example: Illustration: Distinct photography and camping queries can help coverage, but random orthogonal vectors are not useful by themselves.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Core mechanism
Why does EvoPilot treat a matched two-arm comparison, not a successful training job, as evidence?
Verify, Don't Trust: Agentic Model Development for Video Discovery Retrieval at Scale
Show answer and example
Answer: One artifact cannot attribute improvement. EvoPilot matches base, data, evaluator, and output depth, allowing only the approved treatment and checking actual artifact state and provenance. Extra changes invalidate comparison; verification can pass for a negative result and is not release approval.
Illustrative example: A new interaction layer trained twice as long cannot claim sole credit until budgets are matched.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Implementation details
Why read EvoPilot final state and provenance from actual artifacts rather than scheduler success?
Verify, Don't Trust: Agentic Model Development for Video Discovery Retrieval at Scale
Show answer and example
Answer: A job can partially fail, use the wrong config, or leave stale files. Missing final state, version, or actual differences must fail closed. The local check rejects output-depth mismatch, demonstrating validation of execution rather than declarations.
Illustrative example: A completed job with missing metrics cannot reuse last round's JSON as current evidence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Method comparison
Why is reusing a high-quality historical baseline not necessarily a matched treatment comparison?
Verify, Don't Trust: Agentic Model Development for Video Discovery Retrieval at Scale
Show answer and example
Answer: Its base, data, evaluator, or output depth may differ. Match both arms so the approved treatment is the only actual change. More seeds estimate variance but cannot fix a mismatched comparison.
Illustrative example: A model with ten output layers versus a five-layer historical baseline did not only change its interaction head.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Basics
Why can a negative result pass EvoPilot Verify?
Verify, Don't Trust: Agentic Model Development for Video Discovery Retrieval at Scale
Show answer and example
Answer: Verify checks artifact integrity and comparison protocol, not performance direction. A matched, complete regression is valid evidence, and human Review checks interpretation and acceptance. Do not call verification a gain or discard a real negative result.
Illustrative example: An isolated interaction head that lowers hit rate is negative evidence; a gain after also changing data is not attributable.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Core mechanism
Why not retire every teacher once overall student success is close?
RetireOPD: Self-Retiring On-Policy Distillation for Agentic Reinforcement Learning
Show answer and example
Answer: Each skill checks both outcome closeness and gap no longer shrinking. One criterion risks early withdrawal. Local mechanism checks don't reproduce paper teacher training.
Illustrative example: Search may qualify while formatting still lags; one total score can't retire both.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Implementation details
Do local retirement conditions show actual decoupled teachers were trained?
RetireOPD: Self-Retiring On-Policy Distillation for Agentic Reinforcement Learning
Show answer and example
Answer: No. Only retirement state and student update kernel. Log skill identity, success, gap trends and state; registry labels aren't multi-teacher training.
Illustrative example: A synthetic retired state tests a branch, not genuine replacement.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Method comparison
Why use both success and distribution gap for retirement?
RetireOPD: Self-Retiring On-Policy Distillation for Agentic Reinforcement Learning
Show answer and example
Answer: Similar success can leave valuable behavior differences. A stalled gap can occur despite low success. Joint thresholds estimate remaining teaching value, not universal teacher surpassing.
Illustrative example: Matching task scores alone don't show tool choices stopped improving.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Basics
Why make RetireOPD teachers skill-specific and retirable?
RetireOPD: Self-Retiring On-Policy Distillation for Agentic Reinforcement Learning
Show answer and example
Answer: Mastery varies. Retire when student approaches skill teacher and distribution improvement stalls, then explore with RL. Fixed uniform budgets can overteach known skills and underteach weak ones.
Illustrative example: Search and output format can retain different guidance.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Core mechanism
When does On-Demand Attention trigger global retrieval?
On-Demand Attention: Efficient Long-Context Decoding with Learned Recall
Show answer and example
Answer: A cheap recall head predicts benefit before expensive global attention and triggers it only for worthwhile tokens. Full KV stays stored; it saves some reads, not cache capacity. Misses can hide relevant distant evidence.
Illustrative example: Punctuation may use local context, while quoting an opening number triggers global reading; decide before paying for it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Implementation details
Why must ODA decide before computing global attention?
On-Demand Attention: Efficient Long-Context Decoding with Learned Recall
Show answer and example
Answer: Computing global attention first already pays the cost. The gate must control conditional execution. A local A100 receipt verifies the path, not end-to-end vLLM throughput.
Illustrative example: Reading the whole book before choosing the contents page saves no reading.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Method comparison
How does on-demand retrieval differ from evicting old KV?
On-Demand Attention: Efficient Long-Context Decoding with Learned Recall
Show answer and example
Answer: ODA retains all KV and varies when it is read. Eviction removes historical entries from the accessible set and changes memory capacity as well.
Illustrative example: A token skipped now may still be retrieved next step if retained.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Basics
Are ODA local and remote history divided by importance?
On-Demand Attention: Efficient Long-Context Decoding with Learned Recall
Show answer and example
Answer: No. They describe distance relative to the current token. Relevant evidence can be far and nearby words irrelevant, motivating optional global reads.
Illustrative example: A name at the document start can be critical outside the current window.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Core mechanism
Why does bidirectional dQwen3.5 need more than removing an attention mask?
dQwen3.5: Adapting Hybrid Language Models into Bidirectional Diffusion Language Models
Show answer and example
Answer: The hybrid backbone also has recurrence. Both state propagation and attention must support bidirectional context for masked denoising. Running a propagation kernel does not establish a fully adapted checkpoint or loss curve.
Illustrative example: A sentence-start blank can need the end; changing only attention leaves recurrence one-way.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
Implementation details
Why test both attention and recurrence for bidirectional conversion?
dQwen3.5: Adapting Hybrid Language Models into Bidirectional Diffusion Language Models
Show answer and example
Answer: Removing the attention mask can leave recurrence one-way. Local CUDA only confirms component execution, not a 9B checkpoint or full denoising curves.
Illustrative example: Right context reaching one branch does not make the other bidirectional.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
Method comparison
How does masked bidirectional visibility differ from autoregressive training?
dQwen3.5: Adapting Hybrid Language Models into Bidirectional Diffusion Language Models
Show answer and example
Answer: AR is prefix-causal; denoising can use visible left and right context. Adapt the hybrid state path and do not assume causal cache equivalence.
Illustrative example: A known word after a blank is legal evidence for denoising but not next-token AR.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Basics
Why convert a hybrid backbone into a bidirectional diffusion LM?
dQwen3.5: Adapting Hybrid Language Models into Bidirectional Diffusion Language Models
Show answer and example
Answer: Masked denoising uses visible context on both sides rather than causal next-token prediction. Attention and recurrent paths, and the objective, all change.
Illustrative example: A masked middle word can use left and right words.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Core mechanism
Why vary planning, action space, and context separately while fixing the execution loop?
An Empirical Study of Harness Design for Coding Agents
Show answer and example
Answer: Full-harness comparisons change many factors, confounding more reasoning, easier actions, and improved state management. Control components separately under the same loop and evaluate effects and costs. References or statistical contracts do not run the paper's large coding workloads.
Illustrative example: Change only the editor to precise replacement while keeping prompts and budgets fixed, rather than crediting a simultaneous model change to the action space.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Implementation details
Does testing controllable local harness factors mean SWE-bench was run?
An Empirical Study of Harness Design for Coding Agents
Show answer and example
Answer: No. The mini-suite verifies factors and control flow. Keep model version, budget, and tool protocol fixed, and do not report benchmark capability without actual tasks and execution evidence.
Illustrative example: A simulated tool success cannot be converted into a real repository repair rate.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Method comparison
Why not combine more detailed planning and a richer action space into one improvement claim?
An Empirical Study of Harness Design for Coding Agents
Show answer and example
Answer: They affect cost and behavior through different pathways. A combined change does not reveal individual contributions; controlled factors are needed, and interactions may occur. Do not assign a combined gain to one component.
Illustrative example: Success after adding both planning and tools does not tell which helped most.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Basics
What decisions outside the model does the harness encompass in a harness design study?
An Empirical Study of Harness Design for Coding Agents
Show answer and example
Answer: It organizes planning, available actions, and context management and executes tasks through an underlying loop. These are workflow decisions, not model-weight changes. Identifying editable components explains why the same model behaves differently under different execution designs.
Illustrative example: One coding model may see a single file fragment or compressed context from several files. That is a harness change.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why dynamically constrain decoding despite hierarchical ad IDs?
Show answer and example
Answer: Hierarchy organizes ads but not request-specific eligibility. Constrained beams exclude invalid paths during search, while generation/discrimination/ranking learn relevance. Postfiltering can waste beams.
Illustrative example: Block an unavailable ad path before it consumes the candidate budget.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Should ANGLE filter illegal beams after generation or at every expansion?
Show answer and example
Answer: Apply request-specific legality at every step. Global vocabulary is not current eligibility. Select fusion coefficients on validation only.
Illustrative example: Mask a region-ineligible prefix immediately rather than generate ten ads and reject nine.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
How does adding discrimination differ from beam selection by likelihood alone?
Show answer and example
Answer: Likelihood expresses contextual path probability; discrimination measures target specificity and ranking links utility. Tune their trade-off on validation; neither guarantees all metrics improve.
Illustrative example: A popular summary can lose to a more suitable niche path, unless discrimination is overweighted.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Why is intent × summary an ad hierarchy rather than an ordinary ID?
Show answer and example
Answer: Intent groups needs and summary refines meaning, so paths share information. Joint objectives learn predictive and discriminative IDs, but codes still need real eligible ad mapping.
Illustrative example: Generate travel preparation then portable storage, not an unrelated flat ID.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Core mechanism
Why compare token likelihoods rather than rank offline traces by reward?
Trajectory Learnability for Offline On-Policy Distillation
Show answer and example
Answer: Success alone doesn't locate learning opportunities. Train reference on successes and compare likelihoods to current policy to estimate learnability. Depends on reference and offline coverage; local kernel only.
Illustrative example: Reweight two correct traces differently when one is mastered and the other has a difficult turning point.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Implementation details
Does local learnability aggregation include full GPU training?
Trajectory Learnability for Offline On-Policy Distillation
Show answer and example
Answer: No. It implements token aggregation and trajectory weighting. Score identical text/prefix under both versions and record them; don't subtract different responses.
Illustrative example: Old model scoring A and current model scoring B confounds text change.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Method comparison
How does offline reweighting differ from collecting fresh on-policy data?
Trajectory Learnability for Offline On-Policy Distillation
Show answer and example
Answer: Weight allocates budget inside existing coverage. New sampling reaches new states. OPD in the name doesn't make replay current-policy generation.
Illustrative example: Reweighting a cache cannot create an absent solution.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Basics
Why isn't trajectory learnability just success reward?
Trajectory Learnability for Offline On-Policy Distillation
Show answer and example
Answer: Success describes outcome, whereas current student mastery determines learning value. Token likelihood changes between successful reference and current policy reweight offline traces for remaining opportunity.
Illustrative example: Two correct solutions can differ because student already knows one and misses the other's key step.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Core mechanism
How does COMPO infer a direction from paired perturbations, and why threshold it?
A Zeroth-Order Paradigm for LLM Preference Alignment
Show answer and example
Answer: Which of positive and negative perturbations performs better on the same objective gives a directional sign. One-bit feedback is noisy, so coordinate thresholds suppress unreliable components. Objective sampling and evaluation still cost resources.
Illustrative example: Compare small forward/backward perturbations under fixed preferences; leave unreliable coordinates unchanged.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Implementation details
Why threshold each coordinate after a COMPO one-bit update?
A Zeroth-Order Paradigm for LLM Preference Alignment
Show answer and example
Answer: One comparison gives coarse information and weak coordinates may reflect noise. Thresholding suppresses them but can remove small genuine signals. Local tests cover comparison oracle and threshold kernel, not LLM alignment.
Illustrative example: A coordinate consistently near zero may be suppressed, not automatically deemed useless.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Method comparison
What information do COMPO's zeroth-order comparisons and first-order backpropagation need?
A Zeroth-Order Paradigm for LLM Preference Alignment
Show answer and example
Answer: First order needs a differentiable objective and backward access; zeroth order ranks perturbed evaluations. Repeated comparisons still cost compute and noise affects direction; no backward pass does not imply free or more accurate.
Illustrative example: A preference judge can rank two variants without returning gradients for any layer.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Basics
Does COMPO's black-box optimization change prompts instead of weights?
A Zeroth-Order Paradigm for LLM Preference Alignment
Show answer and example
Answer: No. It updates model parameters without backpropagating through the preference objective. It needs parameter perturbation and an objective comparator. Prompt-only remote access does not satisfy that requirement.
Illustrative example: Load and compare two nearby checkpoints; editing chat prompts is another search.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Core mechanism
Why let ASPIRE choose draft lengths independently within a batch?
ASPIRE: Asynchronous Batch Self-Speculative Decoding
Show answer and example
Answer: Requests differ in acceptance and remaining state. One length can waste checks or lose gains. Use batch costs and periodic full-attention refresh for self-speculation. Local tests do not modify a full engine.
Illustrative example: Give different lengths to frequently accepted and first-token-rejected requests, while accounting for shared batch cost.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Does the local ASPIRE scheduler and refresh gate constitute a vLLM integration?
ASPIRE: Asynchronous Batch Self-Speculative Decoding
Show answer and example
Answer: No. Verify real scheduling, cache handoff, refresh execution, and time separately. A reference schedule is not deployment throughput.
Illustrative example: A log showing draft length eight does not prove asynchronous service execution or correct KV transfer.
Sections in the Chinese source: 复现边界. The example illustrates the mechanism.
Method comparison
Is per-request choice the same as ignoring batch and acting greedily?
ASPIRE: Asynchronous Batch Self-Speculative Decoding
Show answer and example
Answer: No. Decisions still use shared batch costs; ignoring resource and synchronization effects changes the scheduler.
Illustrative example: Longer drafts can benefit one request but increase group verification cost.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Basics
Why choose drafts per request?
ASPIRE: Asynchronous Batch Self-Speculative Decoding
Show answer and example
Answer: Online acceptance and remaining conditions vary. Use current batch cost and per-request statistics, with periodic refresh for state correction.
Illustrative example: A frequent long-draft accepter need not match a frequent rejecter's length.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
Why prune leaves first when compressing multi-turn traces instead of deleting the longest message?
Dependency-Aware Trajectory Refinement for Efficient Multi-Turn Agent Fine-Tuning
Show answer and example
Answer: A dependency DAG marks turns supporting the final answer; leaves are more likely to be redundant branches with no downstream dependency. Strict and then relaxed merging reduce repetition while preserving required information flow. The longest message may contain the key tool evidence, so length-only pruning can remove necessary causal context.
Illustrative example: A long log may contain the only error code, while short chat has no dependents. Prune the chat leaf rather than discarding the log because it is longer.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
Does the auditable local DAG include an actual LLM dependency annotator?
Dependency-Aware Trajectory Refinement for Efficient Multi-Turn Agent Fine-Tuning
Show answer and example
Answer: No. It prunes and merges a supplied fixture DAG to verify structure and rejection paths, not automatic semantic annotation. If input omits a key edge, pruning may legally drop a necessary turn. Compression ratios and mechanism passes are not agent fine-tuning quality or paper-level results.
Illustrative example: If a fixture omits 'the answer depends on turn two,' the pruner cannot infer the edge. Correct test graphs do not validate dependency extraction on open trajectories.
Sections in the Chinese source: 核心公式, 复现边界. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
How do leaf pruning and turn merging differ in semantic preservation?
Dependency-Aware Trajectory Refinement for Efficient Multi-Turn Agent Fine-Tuning
Show answer and example
Answer: Pruning removes branches that no longer serve the objective. Merging compactly represents steps that can be expressed together. Strict merging applies tighter constraints; relaxed merging seeks more compression at greater risk. Preserve order and dependency invariants instead of substituting arbitrary summaries.
Illustrative example: Two reads of identical state can be merged. Keep the causal order between a change and verification, not a vague 'handled' summary.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why can't long multi-turn training trajectories simply be truncated by length?
Dependency-Aware Trajectory Refinement for Efficient Multi-Turn Agent Fine-Tuning
Show answer and example
Answer: Tool traces depend on observations, reasoning, and actions. Early or long content may support the final conclusion, and mechanical truncation makes later turns inexplicable. Dependency Refinement retains necessary semantic connections while reducing cost. Annotations may be wrong; shorter traces do not prove more learning or that deleted content never matters.
Illustrative example: If the first turn establishes authorized scope and the final action relies on it, keeping only recent messages makes the action appear unsupported.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Core mechanism
Why use familiarity-based expert selection and assign training samples back to experts?
CERA-MoA: Co-Evolving Router and Agents for Mixture-of-Agents
Show answer and example
Answer: Routing choices and training assignments feed back: assignments shape future expertise. Middle-layer representations estimate familiarity, a cumulative threshold chooses how many experts to use, and targeted training reinforces specialization. Changing only routing while training everyone identically may not sustain distinct expertise.
Illustrative example: Example: Send code cases toward a code-familiar expert and language cases elsewhere. Ambiguous samples may activate more experts rather than use one fixed team.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Implementation details
What does directed training assignment update beyond inference-only routing?
CERA-MoA: Co-Evolving Router and Agents for Mixture-of-Agents
Show answer and example
Answer: CERA-MoA also sends training samples to designated experts, letting routing and capability co-evolve. Selection masks alone train nothing. Local tests cover familiarity routing and a directed-update reference, not paper-scale expert training. Three-seed state statistics do not establish capability evolution.
Illustrative example: Example: A sample assignment to expert A does not show A received a gradient update. Verify parameter changes and independent task results.
Sections in the Chinese source: 核心公式, 复现边界. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Method comparison
How does a cumulative threshold differ from fixed Top-2 expert selection?
CERA-MoA: Co-Evolving Router and Agents for Mixture-of-Agents
Show answer and example
Answer: Top-2 uses the same number per task. Cumulative selection adapts to score concentration: easy cases can use fewer experts and uncertain cases more. Variable counts change cost, so compare quality with total compute rather than infer efficiency from average routed count.
Illustrative example: Example: Use one clearly dominant expert or several with similar scores instead of always calling two.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Basics
Why does CERA-MoA assign samples to experts familiar with them?
CERA-MoA: Co-Evolving Router and Agents for Mixture-of-Agents
Show answer and example
Answer: Calling the same number and set of models on all questions can waste resources and hinder specialization. Familiarity guides selection and training allocation. It is a routing signal, not proof of a correct answer; verify outcomes and total compute.
Illustrative example: Example: Code and math experts may join different samples. Role names alone do not establish effective division of labor.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
Why upweight low-confidence TLM-DRE turns?
Turn-level Multiscale Density Ratio Estimation for LLM Agents
Show answer and example
Answer: Length-normalized reference logp proxies how well a turn is learned. Lower confidence receives greater scale, changing multi-turn density ratios and asymmetric pressure. Low likelihood does not mean wrong. Local diagnostics do not establish generalization.
Illustrative example: Correct a hesitant tool decision more strongly than a confident recap, while recognizing rare valid actions can be unlikely.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
Does the local TLM-DRE kernel demonstrate real multi-turn agent training?
Turn-level Multiscale Density Ratio Estimation for LLM Agents
Show answer and example
Answer: No. It computes reference turn weights and asymmetric token updates without an agent environment. Retain turn memberships; concatenating into one turn changes the method.
Illustrative example: Three tool turns collapsed into a string cannot test the original allocation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
How does asymmetric positive/negative density weighting differ from uniform weights?
Turn-level Multiscale Density Ratio Estimation for LLM Agents
Show answer and example
Answer: Positive and negative evidence reinforce and suppress differently. A common curve removes that design, but asymmetry still needs real multi-turn evaluation.
Illustrative example: Rewarding a correct action and suppressing a wrong continuation need not share weights.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
Why model turns rather than average all tokens in TLM-DRE?
Turn-level Multiscale Density Ratio Estimation for LLM Agents
Show answer and example
Answer: Turns have different roles and lengths. Multiscale weighting preserves dialogue structure; token averaging may let long turns dominate. Actual credit depends on density ratios and feedback.
Illustrative example: A short crucial tool choice should not be drowned out by a verbose explanation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Core mechanism
Why use masked-source token probability changes rather than length-based credit?
TIAO: Token Importance-Aware Policy Optimization for Text Summarization
Show answer and example
Answer: Tokens vary in evidence dependence; paired scoring supplies local clues and shapes both trajectory and token credit. Length doesn't identify provenance. Masking can itself shift distribution, so association is not factual attribution.
Illustrative example: A date and a connective at similar positions need not receive the same evidence credit.
Sections in the Chinese source: 背景与主要改动, 核心公式与实现对应. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Implementation details
Which response should both TIAO views score?
TIAO: Token Importance-Aware Policy Optimization for Text Summarization
Show answer and example
Answer: Use identical response and prefixes under complete/masked source. Otherwise generation changes confound the intervention. Local dependence kernel does not train the paper's 7B model.
Illustrative example: Scoring A with source and generating B without it doesn't isolate source removal.
Sections in the Chinese source: 背景与主要改动, 核心公式与实现对应. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Method comparison
How does TIAO dual-scale credit differ from one summary score?
TIAO: Token Importance-Aware Policy Optimization for Text Summarization
Show answer and example
Answer: Whole-summary score captures overall quality; token dependence redistributes local updates. A good aggregate doesn't supervise every token correctly.
Illustrative example: One hallucinated date in a good summary calls for localized correction.
Sections in the Chinese source: 背景与主要改动, 核心公式与实现对应. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Basics
Why mask the source document in TIAO?
TIAO: Token Importance-Aware Policy Optimization for Text Summarization
Show answer and example
Answer: Summaries should use evidence. Comparing probabilities with and without source locates evidence-sensitive tokens for trajectory/token credit. Dependence is association, not verified truth.
Illustrative example: Removing a relevant sentence can affect a name more than a conjunction.
Sections in the Chinese source: 背景与主要改动, 核心公式与实现对应. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Core mechanism
Why reverse a preference if the rejected answer is factual?
Style-Debiased DPO: Updating LLM Knowledge with Factuality-Aware Synthetic Preference Data
Show answer and example
Answer: Synthetic preferences can mislabel stylistic differences as knowledge errors, causing ordinary DPO to suppress correct expressions. Identify those replies and reverse their preferences with group-ratio weights to cancel style gradients. Reliable factual assessment and grouping are prerequisites.
Illustrative example: If both replies are factual but one is more courteous, directly following the label may teach courtesy rather than knowledge.
Sections in the Chinese source: 背景与主要改动, 核心公式与实现对应. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Implementation details
Do local SD-DPO factual scores establish that a real judge recognizes truth?
Style-Debiased DPO: Updating LLM Knowledge with Factuality-Aware Synthetic Preference Data
Show answer and example
Answer: No. Fixed inputs supply the scores. The code tests reversal and group-balanced loss under given labels, not automated factual judging or knowledge gains.
Illustrative example: If the test declares an answer factual, correct reversal validates the rule, not identification of the fact.
Sections in the Chinese source: 背景与主要改动, 核心公式与实现对应. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Method comparison
How does SD-DPO reversal differ from dropping stylistic pairs?
Style-Debiased DPO: Updating LLM Knowledge with Factuality-Aware Synthetic Preference Data
Show answer and example
Answer: Dropping removes their contribution. Reversing with group-proportion weighting seeks to cancel an existing style direction. Factual correctness must be assessed first, or reversal could promote wrong answers.
Illustrative example: Check both formal and colloquial variants for correctness before balancing their style signals.
Sections in the Chinese source: 背景与主要改动, 核心公式与实现对应. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Basics
Does Style-Debiased DPO update knowledge or enforce one writing style?
Style-Debiased DPO: Updating LLM Knowledge with Factuality-Aware Synthetic Preference Data
Show answer and example
Answer: Synthetic preferences should update knowledge; stylistic differences are confounders. Concision or ornament is not factual truth. That distinction motivates factual checks and group balancing rather than stylistic personalization.
Illustrative example: The same knowledge may be expressed in a sentence or a paragraph; both should stay correct.
Sections in the Chinese source: 背景与主要改动, 核心公式与实现对应. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Core mechanism
Why compare teacher distributions with real and empty visual input?
OPD-Aha: From Linguistic Momentum to Visual Reflection in Multimodal On-Policy Distillation
Show answer and example
Answer: The comparison isolates preference changes attributable to image conditioning despite mistaken text context. It helps separate evidence from prior but doesn't guarantee correctness. Local evidence is reconstruction only.
Illustrative example: A cat image can shift preferences away from the prefix's dog claim relative to no vision.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Implementation details
Does local OPD-Aha reconstruction use a multimodal checkpoint?
OPD-Aha: From Linguistic Momentum to Visual Reflection in Multimodal On-Policy Distillation
Show answer and example
Answer: No. It tests target math. Real implementation needs paired visual/empty inputs and identical student prefixes scored by a model. Fixed differences don't establish visual reflection.
Illustrative example: Manually increasing cat probability checks reconstruction, not image recognition.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Method comparison
How does real/empty vision contrast differ from raising confidence?
OPD-Aha: From Linguistic Momentum to Visual Reflection in Multimodal On-Policy Distillation
Show answer and example
Answer: It changes vision while matching teacher context. Sharpening can amplify language priors. Empty-view design and teacher quality affect attribution.
Illustrative example: If both views confidently say dog, sharpening won't discover the cat.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Basics
What does language inertia mean in OPD-Aha?
OPD-Aha: From Linguistic Momentum to Visual Reflection in Multimodal On-Policy Distillation
Show answer and example
Answer: A wrong student prefix can lead a teacher to keep repeating it despite visual evidence. Compare actual and empty vision to recover visual contributions.
Illustrative example: If prior text says dog and picture shows cat, don't blindly continue dog.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Core mechanism
Why train by actual accepted depth and detach from backbone gradients?
GrowMTP: Can RL Grow Its Own Draft Head?
Show answer and example
Answer: Later drafts only contribute after earlier acceptance, so observed depth reflects use. Decoupling protects reward learning. Hardware and verification costs determine actual speed; local loss isn't acceleration or capability evidence.
Illustrative example: A third draft token isn't as reachable as first, and improving it doesn't prove stronger policy.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Implementation details
Why decouple GrowMTP gradients, and was speed measured?
GrowMTP: Can RL Grow Its Own Draft Head?
Show answer and example
Answer: Keep draft optimization from inadvertently changing main RL objective. Local code tests acceptance-depth loss only, without measured GPU throughput. End-to-end speed needs timing.
Illustrative example: More accurate drafting may also add compute.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Method comparison
How does acceptance-depth learning differ from equal future-head supervision?
GrowMTP: Can RL Grow Its Own Draft Head?
Show answer and example
Answer: A later head helps only after earlier proposals are accepted. Depth coupling reflects use, unlike equal head contributions. Still verify with target.
Illustrative example: Head three saves a step only if the first two proposals pass.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Basics
Why grow draft heads during RL in GrowMTP?
GrowMTP: Can RL Grow Its Own Draft Head?
Show answer and example
Answer: Main policy evolves, making static drafters stale. Train multi-token heads with rollout acceptance to follow policy. Faster drafting isn't task capability improvement.
Illustrative example: If policy learns new phrasing, drafter should propose it to retain acceptance.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Core mechanism
Why propagate teacher credit backward and add verifiable advantage?
Show answer and example
Answer: Later teacher evidence captures effects of early actions; verified outcomes align imitation with task returns. Teacher preference isn't correctness. Local formula checks don't establish capability.
Illustrative example: An early bad choice may cause several later gaps; credit it upstream while still requiring final verification.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Implementation details
What does the local Gamma-OPD kernel verify?
Show answer and example
Answer: Discounted propagation and bounded reward, not full language model training. Preserve order, terminal masks and frozen teacher signals. Shuffling tokens cannot verify temporal credit.
Illustrative example: Moving future credit to another response loses trajectory meaning even when sums are computable.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Method comparison
How does discounted teacher credit differ from adding unbounded outcome reward?
Show answer and example
Answer: Discounting carries sequential distillation; bounded advantage controls strength of verified outcomes. Arbitrary reward weights change the balance without proving one better.
Illustrative example: A giant terminal reward should not overwhelm temporal guidance without limits.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Basics
Why add temporal credit to Gamma-OPD instead of only immediate log gaps?
Show answer and example
Answer: Choices shape future sequence. Discounted future teacher gaps inform current credit, with verifiable return constraining direction. This isn't only immediate imitation.
Illustrative example: An early branch can cause several later divergences that its local log gap underestimates.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Core mechanism
Why conditionally activate Micro Tokens after a Macro Proxy locates a region?
VideoMM: Adaptive Macro-Micro Inference for Efficient Video MLLMs
Show answer and example
Answer: Coarse evidence cheaply removes irrelevant areas but can miss tiny text or movements. Activating detail only when needed focuses compute on uncertainty. This preserves coarse-to-fine evidence access rather than enforcing one low-resolution summary.
Illustrative example: Seeing someone hold up a sign may suffice for a yes/no question, but the number on it can require high-resolution tokens.
Sections in the Chinese source: 背景与主要改动, 核心公式与实现对应. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Implementation details
Does the macro/micro GPU receipt validate the complete video MLLM?
VideoMM: Adaptive Macro-Micro Inference for Efficient Video MLLMs
Show answer and example
Answer: No. A100 executed specific state transitions, not a full video MLLM. Trigger counts, selected indices, and budgets are mechanism evidence, not LongVideoBench, throughput, or verification of semantic consensus.
Illustrative example: Supplying artificial low consensus and reaching Micro verifies branch execution, not estimating consensus from actual footage.
Sections in the Chinese source: 核心公式与实现对应, 本地复现, 复现边界. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Method comparison
What is the trade-off between consensus-triggered Micro and equal resolution budgets?
VideoMM: Adaptive Macro-Micro Inference for Efficient Video MLLMs
Show answer and example
Answer: Fixed budgets may spend on simple regions; insufficient-consensus activation focuses compute but can miss details if routing fails. Compare identical videos, questions, and cost before claiming better QA.
Illustrative example: Broad scene-type questions may not need detail; distinguishing similar tools can need it even when macro evidence seems consistent.
Sections in the Chinese source: 背景与主要改动, 核心公式与实现对应. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Basics
Why preview video coarsely before spending high-resolution budget?
VideoMM: Adaptive Macro-Micro Inference for Efficient Video MLLMs
Show answer and example
Answer: High-fidelity long-video input is costly, and relevance varies across segments and regions. A Macro Proxy supplies location cues, then detail is requested when coarse evidence is insufficient. Localization and answer evidence are not equivalent.
Illustrative example: Locate a person entering a room, then inspect their hand rather than allocating HD tokens to every segment.
Sections in the Chinese source: 背景与主要改动, 核心公式与实现对应. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Core mechanism
Why can the highest-relevance tokens be a poor fixed-budget subset?
StackTok: Accelerating VLMs Inference with Budget-Adaptive Visual Token Selection
Show answer and example
Answer: They may repeat the same region and omit scene structure. StackTok prioritizes relevance but switches criteria when a budget-dependent coverage reference indicates missing support. Coverage is not a fixed decoration; it depends on question and budget.
Illustrative example: Many face patches do not supply the doorway location needed for a layout question.
Sections in the Chinese source: 背景与主要改动, 核心公式与实现对应. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Implementation details
Does passing the local single-crop selector show acceleration in five VLMs?
StackTok: Accelerating VLMs Inference with Budget-Adaptive Visual Token Selection
Show answer and example
Answer: No. It exercises reference-gated selection on one crop, with three-seed fixtures checking ordering, gates, and budgets. No paper VLM is loaded. Correct indices do not establish a shorter forward, latency, or accuracy.
Illustrative example: Ten valid distinct indices meet the selector budget, but downstream still paying for all tokens would save no compute.
Sections in the Chinese source: 核心公式与实现对应, 本地复现, 复现边界. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Method comparison
How does adaptive coverage differ from a fixed mixture of two rankings?
StackTok: Accelerating VLMs Inference with Budget-Adaptive Visual Token Selection
Show answer and example
Answer: Fixed mixing holds one relevance/coverage ratio. StackTok switches criteria when current support is insufficient for the budget. This is a selection-time constraint, not a static diversity score; real fixed-budget VLM evaluation remains necessary.
Illustrative example: With two slots left and missing key regions, fill coverage; once support is sufficient, favor relevance.
Sections in the Chinese source: 背景与主要改动, 核心公式与实现对应. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Basics
Why not keep only the most obviously relevant StackTok positions?
StackTok: Accelerating VLMs Inference with Budget-Adaptive Visual Token Selection
Show answer and example
Answer: Some questions need several objects or a relationship. A single hotspot can consume the budget and omit necessary evidence. Coverage is not uniform sampling or unconditional diversity; the selected set should serve the question.
Illustrative example: To compare two car positions, several patches of the most prominent car are insufficient without the other car.
Sections in the Chinese source: 背景与主要改动, 核心公式与实现对应. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Core mechanism
What path changes when persistent recurrent memory sits between decoder halves?
Persistent Recurrent Memory Between Transformer Layers Improves Language Model Generalization
Show answer and example
Answer: Observe the intermediate state, update persistent state with a GRU, then gate its influence into later layers. This is neither a static prefix nor unconditional addition. Capacity and reset boundaries remain.
Illustrative example: One layer group observes a new constraint and the next can use the updated memory if relevant.
Sections in the Chinese source: 背景与主要改动, 核心公式与实现对应. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Implementation details
Can observe, GRU update, and gated influence be reordered?
Persistent Recurrent Memory Between Transformer Layers Improves Language Model Generalization
Show answer and example
Answer: Not without changing the topology. Specify observation source, old state, and whether later layers read pre- or post-update state; compare a reference. Local checks do not reproduce full TinyStories curves.
Illustrative example: Reading old state before updating differs from influencing with freshly updated state.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Method comparison
How does persistent GRU memory differ from a residual connection?
Persistent Recurrent Memory Between Transformer Layers Improves Language Model Generalization
Show answer and example
Answer: A residual passes representation through the current graph. Observe–update–influence maintains extra state with a separate lifetime and update rule. It is not infinite token storage.
Illustrative example: A persistent state can survive without reconstructing prior inputs word for word.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Basics
What extra state does Persistent Recurrent Memory add?
Persistent Recurrent Memory Between Transformer Layers Improves Language Model Generalization
Show answer and example
Answer: Between layer groups, current representations observe and update a persistent recurrent state that then influences computation through gates. It is distinct from residual accumulation.
Illustrative example: Earlier extracted information can persist and assist the subsequent layer group.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Core mechanism
Why is a seed insufficient for auditing stepwise replay?
OPEN-1B: A Fully Auditable Training Run
Show answer and example
Answer: Batch order, kernel reductions, and collectives affect floating-point trajectories. OPEN-1B records them and per-step state hashes to localize discrepancies. Hashes detect but do not explain differences or guarantee cross-hardware parity.
Illustrative example: Different summation order can cause drift; hashes locate the first differing step.
Sections in the Chinese source: 背景与主要改动, 核心公式与实现对应. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Implementation details
Does matching a canonical hash prove full 400B-token reproduction?
OPEN-1B: A Fully Auditable Training Run
Show answer and example
Answer: No. Local work checks fixed-order reduction and hashing. Full audit requires actual batches, states, and replay; a hash neither executes training nor proves correctness.
Illustrative example: Equal hashes on two small tensors do not replay the paper's entire training.
Sections in the Chinese source: 本地复现, 复现边界. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Method comparison
How does fixing seeds differ from fixing reductions, batches, and communication?
OPEN-1B: A Fully Auditable Training Run
Show answer and example
Answer: Seeds control only part of randomness. Floating reductions, batch order, and collectives must be explicit for replay; equal seeds do not imply bitwise equivalence across hardware.
Illustrative example: Changing addition order can introduce drift despite identical initialization.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Basics
Why audit states inside the training loop?
OPEN-1B: A Fully Auditable Training Run
Show answer and example
Answer: The final checkpoint does not explain its trajectory. Fix data and execution order and record hashes to replay and compare a specific step.
Illustrative example: If drift starts at step 100, inspect its inputs and state instead of only final accuracy.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Core mechanism
Why can LoopSpec use intermediate loop predictions without an independent drafter?
LoopSpec: Pipelined Self-Speculative Decoding for Looped Transformers
Show answer and example
Answer: Shallower loop states give cheap predictions and deeper computation verifies them. Pipelined residual proposals aim to overlap waiting. Actual acceptance and scheduling determine gains; a reference scheduler does not measure GPU throughput.
Illustrative example: Reusing shallow drafts saves a separate model but frequent rejection can erase benefits.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
Implementation details
Does a depth scheduler establish real GPU pipelining?
LoopSpec: Pipelined Self-Speculative Decoding for Looped Transformers
Show answer and example
Answer: No. It checks proposal/depth transitions, not actual overlap, cache handoff, or verification execution. Planned concurrency is not measured throughput.
Illustrative example: A log claiming overlap cannot show synchronization did not serialize the GPU.
Sections in the Chinese source: 本地复现, 复现边界. The example illustrates the mechanism.
Method comparison
What does LoopSpec reuse compared with an independent drafter?
LoopSpec: Pipelined Self-Speculative Decoding for Looped Transformers
Show answer and example
Answer: It reuses intermediate predictions and shared loop structure, rather than a second parameter set and path. Speed still depends on proposal quality, verification, and scheduling.
Illustrative example: Removing separate drafter weights does not ensure candidates are accepted.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Basics
Why is self-speculation natural for shared-block loops?
LoopSpec: Pipelined Self-Speculative Decoding for Looped Transformers
Show answer and example
Answer: Intermediate depth provides draft predictions and deeper loops verify. This reuses model weights; pipelined residual proposals reduce intended waiting.
Illustrative example: Draft with fewer loops, then verify with more.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Core mechanism
Why is λ=0 still calibrated in interpolated null calibration?
How Calibration Content Shapes Attention-Based Reranking
Show answer and example
Answer: Zero removes the instruction-null contribution but still subtracts bare-null. Align all three score vectors by document and select λ on validation. A global oracle weight is not held-out gain. Local work is a score kernel.
Illustrative example: A long document can score highly without a query; bare-null still removes this baseline.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Implementation details
Can equal-length score arrays be subtracted directly?
How Calibration Content Shapes Attention-Based Reranking
Show answer and example
Answer: Document IDs and ordering must match, not just shape. Choose λ on validation, not a full-data oracle. The local score operator is not a real attention extractor.
Illustrative example: If pass two swaps docs A and B, positional subtraction applies the wrong null score.
Sections in the Chinese source: 背景与主要改动, 本地复现. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Method comparison
What do standard ICR, bare-null, and no calibration subtract?
How Calibration Content Shapes Attention-Based Reranking
Show answer and example
Answer: λ=1 subtracts instruction null; λ=0 subtracts bare null; only uncalibrated subtracts neither. Document all three scoring inputs and subtraction terms.
Illustrative example: A nonzero s(empty,empty) can make λ=0 rank differently from raw s(q,I).
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Basics
Why compare instruction null with bare null in attention reranking?
How Calibration Content Shapes Attention-Based Reranking
Show answer and example
Answer: Instructions can affect scores even without a query. Interpolation separates that influence and controls which baseline is subtracted from query-plus-instruction scores.
Illustrative example: “Judge relevance” may raise document scores without conveying the user's query.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why not use ECHO bonus logits as the final sampling distribution?
ECHO: Early-layer Collaborative Hierarchical Orchestration with Bonus Logits in Speculative Decoding
Show answer and example
Answer: Frequent cheap early proposals and less frequent authoritative late checks cooperatively expand the candidate tree. Exploration may differ from the target and requires exact rejection correction. Local candidate/tensor checks do not measure throughput.
Illustrative example: A bonus branch may enter verification without being accepted automatically.
Sections in the Chinese source: 背景与主要改动, 核心公式与实现对应. The example illustrates the mechanism.
Implementation details
How does candidate merge validation differ from CUDA throughput evidence?
ECHO: Early-layer Collaborative Hierarchical Orchestration with Bonus Logits in Speculative Decoding
Show answer and example
Answer: Reference verifies merge and lossless correction; A100 receipt verifies the tensor path. Real speed needs early/late scheduling, caching, and measured time.
Illustrative example: Distribution parity does not prove ideal GPU overlap.
Sections in the Chinese source: 本地复现, 复现边界. The example illustrates the mechanism.
Method comparison
Why separate exploration scores from target sampling?
ECHO: Early-layer Collaborative Hierarchical Orchestration with Bonus Logits in Speculative Decoding
Show answer and example
Answer: Bonus logits serve proposals and expansion. Exact rejection correction maintains the target distribution. Sampling bonuses directly changes behavior.
Illustrative example: Trying a favored candidate first does not entitle it to higher final probability.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Basics
Why explore frequently in early layers but verify less often at the end?
ECHO: Early-layer Collaborative Hierarchical Orchestration with Bonus Logits in Speculative Decoding
Show answer and example
Answer: Early proposals are cheap; full depth verifies reliably. Dual loops and bonus logits expand candidates while aiming to reduce waits and preserve corrected sampling.
Illustrative example: Let the light path expand promising branches for full-depth checking.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Core mechanism
Why can't inner harness evolution and outer policy updates be reduced to repeatedly rewriting a prompt?
ScienceBuddy: Recursive-in-Recursive Self-Improvement for Interactive Scientific Agents
Show answer and example
Answer: The two levels change different objects. The inner loop optimizes interaction, while the outer loop uses successful trajectories to update the scientific agent's parameterized policy, which affects the next inner search. Actual execution, filtering, and training must close the loop to demonstrate recursion within recursion. Reference gates or synthetic statistics cannot replace the models and environments involved.
Illustrative example: Find a better experimental checking process, then train the agent on successful trajectories so it uses that process better next time. Rewriting the prompt three times has no second-layer update.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Implementation details
Does the local double-state reference include a lab and real scientific policy learning?
ScienceBuddy: Recursive-in-Recursive Self-Improvement for Interactive Scientific Agents
Show answer and example
Answer: No. It runs a reference two-level state update and gate, without a real laboratory, full scientific tasks, or paper-scale training. Three-seed intermediate values verify branches and information flow, not reproducible scientific discovery rates. Counts of state updates do not validate full recursive learning.
Illustrative example: Two inner changes and one outer update show control-state movement. Without real trajectories and gradients, they do not show that the policy learned a task.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Method comparison
What trade-offs arise from two-level feedback versus single-level training with a fixed harness?
ScienceBuddy: Recursive-in-Recursive Self-Improvement for Interactive Scientific Agents
Show answer and example
Answer: Two levels can improve both data generation and policy, but simultaneous changes complicate attribution and introduce nonstationarity. Record versions, trajectory provenance, and updates separately. A fixed harness is easier to control but can constrain exploration. Mechanism comparisons do not replace scientific-task or experimental outcomes.
Illustrative example: A higher outer-loop score might come from better tool feedback or from new parameters. Without versions, the contributions are hard to distinguish.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Basics
Why does ScienceBuddy put tool interaction and policy learning into one continuing process for scientific tasks?
ScienceBuddy: Recursive-in-Recursive Self-Improvement for Interactive Scientific Agents
Show answer and example
Answer: Scientific tasks involve proposing actions, using tools, interpreting observations, and adjusting plans. A fixed prompt is unlikely to handle ongoing feedback. ScienceBuddy studies improving workflows and policies together during interaction. Track the two levels separately; simulated state updates are not evidence of a real scientific environment or model training.
Illustrative example: A search finds the original hypothesis lacks evidence, so the workflow can require source checks first. Whether the policy learned that judgment needs validation on subsequent real tasks.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Core mechanism
Why refresh visual topology and the precise text index together?
RepoAtlas: Guiding Coding Agents via Evolving Multimodal Repository Views
Show answer and example
Answer: Topology shows module dependencies, and text locates actual editable symbols. If views disagree, a correct graph can point to an obsolete file. Select–project–refresh chooses task-related subgraphs under budget and keeps both views grounded in common sources. A diagram does not replace source evidence.
Illustrative example: Example: After moving a function, update its graph edges and indexed location; refreshing only one can leave nonexistent paths.
Sections in the Chinese source: 背景与主要改动, 核心公式与实现对应. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Implementation details
Does the local bounded text view implement a real multimodal SWE-bench repository environment?
RepoAtlas: Guiding Coding Agents via Evolving Multimodal Repository Views
Show answer and example
Answer: No. Public observations produce a budgeted text view and selection/refresh state. It does not claim real browser or SWE-bench integration. Indexes, edges, and budgets are checkable; visual topology generation, model consumption, and repair outcomes need separate execution evidence.
Illustrative example: Example: Ten nodes and source locations show view data exists, not that a model viewed a graph and fixed an issue.
Sections in the Chinese source: 核心公式与实现对应, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Method comparison
What is the trade-off between a task-related code subgraph and a whole-repository summary?
RepoAtlas: Guiding Coding Agents via Evolving Multimodal Repository Views
Show answer and example
Answer: A subgraph keeps relevant relations within budget but can miss remote dependencies. A whole summary covers more at lower detail. Refresh selection as implementation and feedback change. A one-time snapshot or visible graph is not improved code understanding.
Illustrative example: Example: Editing authentication needs callers and configuration. If initially omitted, failed tests can justify expanding the subgraph.
Sections in the Chinese source: 背景与主要改动, 核心公式与实现对应. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Basics
What map does RepoAtlas build first for an unfamiliar large repository?
RepoAtlas: Guiding Coding Agents via Evolving Multimodal Repository Views
Show answer and example
Answer: Reading files one at a time can lose global structure, while a broad overview cannot locate symbols. RepoAtlas combines topology and precise source entry points so agents can understand dependencies and inspect code. Navigation does not replace source evidence, and stale structure can misdirect later edits.
Illustrative example: Example: For login bugs, trace authentication and API dependencies before opening the validation function. Module names do not reveal the real conditions.
Sections in the Chinese source: 背景与主要改动, 核心公式与实现对应. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why jointly learn a write Planner and read Trigger from delayed cross-session reward?
Interactive Memory Learning for Long-Term Conversations
Show answer and example
Answer: Some details reveal their usefulness only in later sessions. Retrieval timing determines whether they affect results. Shared delayed reward links writing to later use instead of rewarding summary quality alone. Evaluate in chronological order; future questions must not preselect what to write.
Illustrative example: Example: Saving a changed workplace matters for commute planning next week. Same-day chat fluency may never reward saving it.
Sections in the Chinese source: 背景与主要改动, 核心公式与实现对应. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
Why distinguish information available at writing time from later evaluation in an observation-safe state machine?
Interactive Memory Learning for Long-Term Conversations
Show answer and example
Answer: The writer sees observations available then. Future questions, answers, and scores can supply external reward only after the corresponding execution. Local Planner/Trigger diagnostics verify isolation and feedback, not paper-scale training or learned preferences from preset fixture rewards.
Illustrative example: Example: Hiding day three's question in day one's write input makes fact selection easy, but this lookahead leak invalidates the learning claim.
Sections in the Chinese source: 核心公式与实现对应, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
How does delayed cross-session reward differ from reward based on per-turn retrieval similarity?
Interactive Memory Learning for Long-Term Conversations
Show answer and example
Answer: Similarity is an immediate proxy that may not predict later use. Delayed reward observes later dialogue outcomes to learn long-term writing and trigger value. It introduces credit assignment; revealing future questions to the writer would no longer test learning for an unknown future.
Illustrative example: Example: An allergy detail may be unrelated to the present topic yet crucial for ordering later. Similarity can underestimate that long-term value.
Sections in the Chinese source: 背景与主要改动, 核心公式与实现对应. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Why is it hard for a cross-session assistant to know on day one what will matter on day three?
Interactive Memory Learning for Long-Term Conversations
Show answer and example
Answer: Future requests have not occurred. The writer chooses likely durable facts from observed conversation, and the reader retrieves them in suitable contexts. Interactive Memory learns from actual future usefulness rather than giving the writer future questions.
Illustrative example: Example: A user casually mentions avoiding peanuts on day one and asks for meal planning on day three. The value of that note becomes evident later.
Sections in the Chinese source: 背景与主要改动, 核心公式与实现对应. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Core mechanism
Why are verifiable hidden motives better for social-reasoning evaluation than judging the fluency of retellings?
Verifiable Social Reasoning for LLM Assistants
Show answer and example
Answer: Retellings can be biased or incomplete. Fluent replies may not identify real goals. A simulator keeps motives and event conditions hidden from the actor and uses them to check inferences rather than textual agreement. Only the verifier sees references, and fixed fixtures are not actual multi-agent semantic interaction.
Illustrative example: A user calls a coworker's refusal hostility while the simulated cause is a resource conflict. Evaluate whether the assistant distinguishes them from visible evidence rather than echoing the framing.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Implementation details
What boundaries did the local FUSE evaluator execute, and did it call twelve external LLMs?
Verifiable Social Reasoning for LLM Assistants
Show answer and example
Answer: It runs only an observation-safe evaluation state machine, not the external model matrix. Project public observations separately from private motives; never pass an internal object containing gold to the actor callback.
Illustrative example: The grader may know the actual relationship without exposing that field in the assistant input.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Method comparison
How does a framing-bias evaluation differ from ordinary QA accuracy?
Verifiable Social Reasoning for LLM Assistants
Show answer and example
Answer: Framing can influence attribution even with unchanged facts. Comparing retellings against hidden state measures shifts in social judgment. Small simulated state machines do not establish comprehensive real-world social understanding.
Illustrative example: Describing the same lateness as indifference or a traffic delay can change the assistant's judgment.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Basics
Does FUSE train a social agent or provide a verifiable social-reasoning evaluation environment?
Verifiable Social Reasoning for LLM Assistants
Show answer and example
Answer: It is an evaluation: a multi-agent simulation with verifiable hidden motives tests how assistants interpret user retellings and social context. Hidden state supplies grading evidence, not assistant answers. Implementing the simulator or local kernel does not train a stronger social reasoner.
Illustrative example: The simulation knows a character's true intent, while the assistant judges from the retelling. The evaluator checks afterward without revealing it first.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why is changing only the responsible GUI skill component more auditable than regenerating the whole skill?
Reflect, Revise, Reuse: Training-Free Skill Evolution for GUI Agents
Show answer and example
Answer: A local edit ties failure evidence to one component and preserves other verified behaviors. This makes it easier to tell whether the patch fixed the defect without broad drift. A critic needs information isolation, not hidden answers. Verify the revised skill before reusing it elsewhere.
Illustrative example: Example: If login fails only when closing a pop-up, first fix that locator rather than rewrite username entry and submission too.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
Can the local public-observation patch demo establish cross-task reuse on real GUIs?
Reflect, Revise, Reuse: Training-Free Skill Evolution for GUI Agents
Show answer and example
Answer: No. It reads public evidence and applies bounded reference edits without MobileWorld, AndroidWorld, or OSWorld. It verifies no gold access, patch scope, and state transitions. Real screen understanding, clicking, and validated cross-task success remain unestablished.
Illustrative example: Example: Updating a locator after a button name changes in a string verifies the patch contract, not finding that button in an actual screenshot.
Sections in the Chinese source: 核心公式, 复现边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
How does local revision by an isolated critic differ from ad hoc self-editing by the executor?
Reflect, Revise, Reuse: Training-Free Skill Evolution for GUI Agents
Show answer and example
Answer: The critic independently attributes failure from permitted evidence, limits patch scope, and verifies before reuse. Ad hoc edits can carry over the executor's wrong assumptions. Isolation controls evidence and responsibility rather than guaranteeing correctness; retain failure evidence and validation results.
Illustrative example: Example: The executor assumes a page finished loading. A critic can identify a missing wait from observations instead of accepting that the button is absent.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Why can GUI skills become invalid as interfaces and tasks change?
Reflect, Revise, Reuse: Training-Free Skill Evolution for GUI Agents
Show answer and example
Answer: Button positions, page sequence, and preconditions can change. A skill that worked before need not execute in a new state. EvoSkill-GUI studies converting observed failures into verified improvements for later reuse. Training-free still requires real observations and execution checks.
Illustrative example: Example: A top-right button moves into a menu. Replaying the old trace fails, so revise the corresponding step from the current screen.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why keep a ranking residual adapter alongside LazFormer's transferable pretrained backbone?
Show answer and example
Answer: Generative pretraining learns broad sequence patterns, but industrial ranking needs candidate interaction and frequent updates. A residual adds ranking-specific behavior without wholesale overwrite; dense recent and sparse distant tokens with sparse attention control long-history cost. Local deterministic mapping and adapter composition do not reproduce full generative pretraining.
Illustrative example: Illustration: The backbone knows long-term consumption, while a promotion needs extra price interactions. A fixed weighted scorer cannot prove the original training pipeline ran.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why must the LazFormer local implementation not report the paper's two-week A/B numbers as its own training results?
Show answer and example
Answer: The local version has deterministic candidate scoring, transferable mapping, and a ranking residual with fusion selected on validation. It lacks full large-model training and online serving. Report the negative public-protocol result and separately cite the paper's business gains.
Illustrative example: Illustration: A residual in local movie scores does not imply generative pretraining; copying the paper's GMV percentages into local metrics would mix evidence sources.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
What long-range information do dense recent and sparse distant tokens retain versus fixed recent truncation?
Show answer and example
Answer: Truncation drops old history entirely. The mixed representation keeps coarse older evidence and detailed recent events, and sparse connections lower pairwise cost. Compression can lose rare events. The deterministic mapping and residual in the local version are interfaces, not original long-sequence pretraining.
Illustrative example: Illustration: Keep every click from the last two days but summarize last year's history by groups. This remembers more than ten recent clicks, yet an important old exception may be lost.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why distinguish transferable generative pretraining from the ranking residual adapter in LazFormer?
Show answer and example
Answer: Pretraining learns general sequence and semantic structure. Ranking needs candidate value and serving objectives; the residual adds a task-specific update without rewriting the base. Dense recent and sparse distant inputs and sparse attention separately control history cost. Verify training and serving mechanisms individually.
Illustrative example: Illustration: Pretraining learns a camera-to-lens progression; adaptation adds the current candidate's price, a business trade-off not necessarily learned by transfer alone.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why does GESE explore diverse faithful titles and then select one for current context?
Show answer and example
Answer: One fixed title lacks personalization; online free generation costs compute and can misrepresent content. Reusable candidate creation and real-time selection are separate, and diversity cannot justify false claims. Fixed-candidate local scoring verifies selection only.
Illustrative example: A documentary can have faithful titles about photography or natural history, but not invented action scenes for clicks.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Why is GESE's local candidate scorer not full LLM generation training?
Show answer and example
Answer: It uses deterministic relevance, diversity, faithfulness, and contextual selection to validate the selection mechanism, not full set generation or production post-training. Report the implemented operators and negative results; paper online metrics are not local measurements.
Illustrative example: Selecting among five hand-written titles verifies a selector, not a model's ability to generate those faithful titles.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
How does GESE allocate compute compared with using an LLM to write each user's title in real time?
Show answer and example
Answer: Reusable candidates amortize expensive generation across requests and leave real-time adaptation to a lightweight selector. Per-user generation is more flexible but adds latency, repeated cost, and consistency risks. Pre-generation limits coverage; deterministic local scoring does not establish production speed or quality.
Illustrative example: Generate five faithful angles once and let a thousand users run only selection. An angle outside the pool cannot be created by the selector.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why separate GESE title exploration from personalized selection?
Show answer and example
Answer: Generate a relevant, diverse, faithful pool, then choose by current user and context. The selector cannot repair a pool lacking faithful options, and catchy wording does not imply satisfaction.
Illustrative example: Title variants can emphasize method, setting, or conclusion, but none may invent results absent from the article.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
How can fixed registers support cross-chunk reasoning after text removal, and what is lost?
Register Tokens for Bounded-State Reasoning in Diffusion Language Models
Show answer and example
Answer: Continuous hidden registers carry compressed previous-chunk information at fixed size. They do not preserve original text or guarantee fine detail. Train and evaluate retention; no complete local diffusion checkpoint.
Illustrative example: A relationship can survive as state while an omitted name cannot be guaranteed recoverable.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
Implementation details
How verify old text is not retained through an undeclared bypass?
Register Tokens for Bounded-State Reasoning in Diffusion Language Models
Show answer and example
Answer: Check that the next chunk receives only the agreed registers, clear old text and undeclared caches, and verify state updates. Local work omits LLaDA/Dream and diffu-GRPO.
Illustrative example: Fixed register count is insufficient if the old text is secretly concatenated anyway.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
Method comparison
How does register carry differ from keeping all text or KV?
Register Tokens for Bounded-State Reasoning in Diffusion Language Models
Show answer and example
Answer: Carry is a fixed-size continuous summary. Text/KV grows and permits finer position access. Fixed registers are not lossless compression.
Illustrative example: Two hundred pages still compress into eight vectors without expanding capacity.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Basics
Why use a fixed number of register tokens across diffusion chunks?
Register Tokens for Bounded-State Reasoning in Diffusion Language Models
Show answer and example
Answer: Hidden states pass some earlier information after text is cleared, while bounding capacity. This is also an information bottleneck.
Illustrative example: The next chunk gets several continuous vectors rather than all previous words.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Core mechanism
Why keep representative queries from several agent phases for KV eviction?
AgentKV: Phase-Aware KV Eviction for Agentic LLMs
Show answer and example
Answer: Planning, tools, and reporting need different evidence. Recent queries alone may underrate material needed later. Phase representatives broaden retention scoring but cannot recover deleted KV or guarantee representativeness.
Illustrative example: Tool arguments dominate recent access, but the final summary still needs the original task constraints.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Implementation details
Does local phase-aware scoring modify SGLang's real KV manager?
AgentKV: Phase-Aware KV Eviction for Agentic LLMs
Show answer and example
Answer: No. It is a scoring reference with phase and cache statistics, without SGLang or CUDA kernel changes. Real eviction, indexing, and execution still need verification.
Illustrative example: Retained indices alone do not prove GPU memory was freed and read correctly.
Sections in the Chinese source: 本地复现, 复现边界. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Method comparison
How does a phase-aware buffer differ from a recent window?
AgentKV: Phase-Aware KV Eviction for Agentic LLMs
Show answer and example
Answer: A recent window uses time. Phase-aware scoring compares history with representatives from multiple phases. It may preserve older evidence but is limited by budget and representative quality.
Illustrative example: An old plan outside the window can still be retained when a phase query matches it.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Basics
Why use agent phase in KV eviction?
AgentKV: Phase-Aware KV Eviction for Agentic LLMs
Show answer and example
Answer: Different phases read different history. Representatives help assess future usefulness beyond current few accesses.
Illustrative example: An early plan can regain importance during final reporting.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Core mechanism
Why pair early stopping and restart from weakest correct turn?
Know When to Stop, Where to Restart: Accelerating Multi-Turn Agentic On-Policy Distillation
Show answer and example
Answer: Teacher cumulative confidence avoids spending on low-value continuations. A weak but correct cached turn reassigns budget to practice. These are distinct choices, not jumps to gold. No real-model throughput validation.
Illustrative example: Stop an uninformative continuation, then retry from an earlier barely successful turn rather than repeatedly replay the familiar beginning.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Implementation details
Can STRIDE restore any environment from cached prefix text?
Know When to Stop, Where to Restart: Accelerating Multi-Turn Agentic On-Policy Distillation
Show answer and example
Answer: Align cached prefix with executed actions and observations. Text alone may omit environment state. Local state machine isn't LLM rollout throughput.
Illustrative example: Do not duplicate a tool side effect on restore and call it the same state.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Method comparison
How does early stopping differ from uniform truncation?
Know When to Stop, Where to Restart: Accelerating Multi-Turn Agentic On-Policy Distillation
Show answer and example
Answer: Teacher cumulative confidence triggers a state-dependent stop paired with prefix restart. A fixed length limit changes coverage differently; shortness doesn't guarantee completion or savings.
Illustrative example: A solved trace and exploratory trace need not stop at the same token count.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Basics
Why restart STRIDE-OPD from the weakest correct turn?
Know When to Stop, Where to Restart: Accelerating Multi-Turn Agentic On-Policy Distillation
Show answer and example
Answer: A cached learnable state focuses budget on barely mastered turns rather than regenerating familiar prefix. Correct qualification ensures a valid starting point.
Illustrative example: Retry from a second turn that only just succeeds once the first is stable.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
Why match output-class frequencies instead of just raising token entropy?
MaD-RL: Matching Distributions for Calibrating LLMs with Reinforcement Learning
Show answer and example
Answer: Randomized words can collapse to the same decision. Map outputs to classes and reward empirical-to-target similarity plus correctness. Hold frequencies fixed per update; unsampled classes get no fictitious on-policy examples. Specify invalid-output treatment.
Illustrative example: Check decisions directly even when language appears diverse.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
Can invalid outputs be dropped from MaD-RL frequency denominators?
MaD-RL: Matching Distributions for Calibrating LLMs with Reinforcement Learning
Show answer and example
Answer: Not arbitrarily; doing so changes empirical frequencies and rewards. Treat frequencies as fixed per update, smooth zero target probabilities explicitly, and do not invent gradients for unsampled categories. Local diagnostics aren't full training.
Illustrative example: Dropping all unclassifiable samples can make remaining frequencies appear closer to target.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
What tradeoff does distribution matching add beyond maximizing accuracy?
MaD-RL: Matching Distributions for Calibrating LLMs with Reinforcement Learning
Show answer and example
Answer: Accuracy can favor easy classes, while matching moves mass toward target frequencies. Report both: matching doesn't guarantee correctness and high accuracy doesn't guarantee target ratios.
Illustrative example: Balanced classes can still include wrong replies in one category.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
Why target output classes rather than token entropy in MaD-RL?
MaD-RL: Matching Distributions for Calibrating LLMs with Reinforcement Learning
Show answer and example
Answer: Classes measure semantic result frequencies; token entropy measures surface uncertainty. Many wordings can yield one class. Combine empirical-to-target class mismatch with correctness.
Illustrative example: Many phrasings recommending the same item category do not improve category coverage.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Core mechanism
Why isn't self-generated data-free OPD independent of prior knowledge?
Data-free On-policy Distillation
Show answer and example
Answer: Teacher parameters, sampling and filters determine curriculum even without a separate task bank. Generated tasks can be repetitive/unreliable. Local diagnostics have not generated a natural-language dataset.
Illustrative example: Generating code exercises avoids human curation but repeated loop problems do not teach omitted concurrency.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Implementation details
Has local DF-OPD generated a real natural-language training set?
Data-free On-policy Distillation
Show answer and example
Answer: No. Only policy-feature diagnostics for generated questions. Need actual generation, filtering and distillation provenance, not changing candidate indices.
Illustrative example: A feature ID isn't a verified math problem.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Method comparison
How does generated curriculum trade off with fixed public tasks?
Data-free On-policy Distillation
Show answer and example
Answer: Generated tasks can target teacher skills but repeat or omit topics. Fixed banks have auditable coverage and collection cost. Validate quality; linguistic format isn't task validity.
Illustrative example: A teacher repeating linear equations does not cover geometry automatically.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Basics
What dependency does data-free OPD reduce?
Data-free On-policy Distillation
Show answer and example
Answer: Teacher-generated prompts reduce collection of a separate post-training problem bank. Teacher knowledge and sampling still provide information.
Illustrative example: Teacher can create exercises without a human bank using existing learned skills.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
Why schedule multi-harness training using both learnability and transferability?
Show answer and example
Answer: An easy harness may teach only its specific interface; a transferable but currently hard one may need exploration. Combining both signals and a bonus allocates limited budget and avoids overfitting to easy scores. Scheduling weights are training decisions; a reference function or multi-seed diagnostics are not cross-harness ability evidence.
Illustrative example: A provides many hints and B resembles deployment. Do not give A all budget for faster near-term gains without checking whether the learning transfers.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
Do local online harness-selection counts establish completed multi-harness LLM RL?
Show answer and example
Answer: No. The reference runs gates, allocation, and state updates, without real LLM RL, full rollouts, or transfer benchmarks. Synthetic learnability and transferability check weight response, not portable capability. Three-seed scheduling statistics cannot establish training effects.
Illustrative example: More allocations to an arm with a supplied high transfer score verify the scheduler. A manually set score does not measure model transfer.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
What trade-offs distinguish uniform multi-harness allocation from bandit scheduling?
Show answer and example
Answer: Uniform allocation does not adapt to changing payoffs. A bandit can sample more from environments worth learning or uncertain at the current stage. It may favor short-term proxies and spend budget on exploration, so assess scheduling cost and transfer rather than only within-harness efficiency.
Illustrative example: After one harness saturates, shift budget to paths with transferable gains, while occasionally revisiting it to guard against stale estimates.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why might learning in one harness fail to improve performance under other interfaces?
Show answer and example
Answer: Tool protocols, prompts, and workflows change which states and actions are encountered. Some learning is specific to that organization. Harness Bandit allocates budget across harnesses for both learning and transfer. In-training gains are not cross-interface generalization; report evaluation conditions separately.
Illustrative example: A model familiar with one edit command may face different arguments elsewhere. More practice on the known interface can improve its score without improving transfer.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Core mechanism
Why match multiple positives in a MIMA request one-to-one with interests instead of independently choosing the closest?
MIMA: Multi-Interest Recommendation via Multi-Positive Exclusive Assignment
Show answer and example
Answer: Independent selection can send all gradients to one interest. MIMA forms an interest–positive similarity cost matrix and finds a globally optimal one-to-one assignment, giving distinct interests supervision. Activation probabilities then calibrate channels. Specialization comes from set supervision, not just more vectors.
Illustrative example: Illustration: Running shoes and a hiking pack can both choose one sports channel under independent matching. A one-to-one assignment makes two channels explain the positives separately.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Implementation details
When do exhaustive search and Hungarian matching optimize the same MIMA objective, and what cannot replace them?
MIMA: Multi-Interest Recommendation via Multi-Positive Exclusive Assignment
Show answer and example
Answer: Both solve the same cost matrix's optimal one-to-one assignment; exhaustive search can verify small cases. Greedy matching is generally not equivalent. Use aligned interest–positive similarities and separately calibrate channels with log activation probability. Plausible pairs do not establish optimality.
Illustrative example: Illustration: A greedy first positive can take the only good slot for the second. Exhaustive enumeration can find the lower-cost allocation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Method comparison
How does one-to-one MIMA matching avoid crowding one channel compared with independently taking the nearest interest?
MIMA: Multi-Interest Recommendation via Multi-Positive Exclusive Assignment
Show answer and example
Answer: Independent argmax can give all positives to one interest and starve the others. Global assignment distributes slots to minimize total cost. Define unmatched handling when counts differ; forced matching can also give a noisy positive its own channel and does not ensure semantic independence.
Illustrative example: Illustration: Two items slightly prefer interest one. One-to-one matching evaluates the total cost of assigning either to interest two, unlike greedy independent matching.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Basics
Why treat positives in one MIMA request as a set rather than imposing an order?
MIMA: Multi-Interest Recommendation via Multi-Positive Exclusive Assignment
Show answer and example
Answer: Coexisting interests need not be assigned by artificial order. Set matching is permutation-insensitive and encourages specialization; routing estimates actual activation. Historical encoding can still be causal; only positive supervision changes.
Illustrative example: Illustration: A request can like shoes and a pack without defining which to learn first or letting the strongest sports channel explain both.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why must a Chronicle Token have a time anchor and summarize only preceding events?
ChronicleRec: Pre-training Temporally Anchored Tokens for Lifelong User Modeling
Show answer and example
Answer: Without the boundary, an earlier representation could include future events, violating temporal causality and making cache meaning unclear. ChronicleRec compresses different ranges, anchors queries on the timeline, reads earlier events only, and masks recent behavior in multiple ways to align short-term intent. Anchored candidate-independent summaries are reusable across candidates.
Illustrative example: A Wednesday summary must not include Friday's camera purchase when scoring Thursday lens recommendations.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why normalize Chronicle Token pooling by valid mask count and read at the query position?
ChronicleRec: Pre-training Temporally Anchored Tokens for Lifelong User Modeling
Show answer and example
Answer: Padding should not shrink short histories; masks and a safe denominator average real events only. Read after causal encoding at the query anchor so the state summarizes only earlier events. Other positions and padded events can alter semantics without causing shape errors.
Illustrative example: A four-slot window with two real actions divides by two. Put the query after both to summarize both; a query between them cannot see the later one.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does ChronicleRec alignment pretraining differ from ordinary next-item prediction?
ChronicleRec: Pre-training Temporally Anchored Tokens for Lifelong User Modeling
Show answer and example
Answer: Branches hide different lengths of recent actions so compressed older history must predict recent intent without nearby evidence. Ordinary next-item prediction may lean on adjacent actions. Do not return hidden targets to evaluation inputs, or the compression test loses its meaning.
Illustrative example: Hide the latest three parenting views and predict the new interest from older history instead of guessing the next item from the last parenting item.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why does ChronicleRec represent near, middle, and distant history instead of a few unordered interest tokens?
ChronicleRec: Pre-training Temporally Anchored Tokens for Lifelong User Modeling
Show answer and example
Answer: Multi-scale pooling preserves temporal structure. Query anchors delimit the past being summarized so stable preferences and recent changes remain distinct. Candidate-independent Chronicle Tokens can be cached, but public compression can still discard target-specific details.
Illustrative example: A long-term science-fiction preference and this week's parenting interest mean more with time scales than as two unordered keywords.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Core mechanism
How does SAS train its selector despite nondifferentiable top-k indices?
SAS: Simple Attention Sparsification via End-to-End Optimization of Context Ranking
Show answer and example
Answer: It preserves continuous gates for selected blocks and adds log gates inside attention softmax. LM loss differentiates through gate values, not indices. History gates normalize jointly and the current block stays at one.
Illustrative example: Changing two selected gates from 0.8/0.2 to 0.6/0.4 changes output and gradients without changing indices.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Implementation details
Where does SAS add the log gate?
SAS: Simple Attention Sparsification via End-to-End Optimization of Context Ranking
Show answer and example
Answer: Add log gates for selected history to attention logits before shared softmax. Normalize history gates together and set current block to one. Post-output scaling changes the normalization relationship.
Illustrative example: Current gate one adds zero; history blocks compete with their respective log gates.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Method comparison
How does LM-loss selector training differ from distilling dense attention rankings?
SAS: Simple Attention Sparsification via End-to-End Optimization of Context Ranking
Show answer and example
Answer: Token prediction provides task feedback rather than supervising imitation of dense attention. It needs continuous gates; hard top-k remains nondifferentiable.
Illustrative example: A block ranked highly by the dense model need not contribute equally to prediction loss.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Basics
Why keep continuous SAS gates rather than only selected indices?
SAS: Simple Attention Sparsification via End-to-End Optimization of Context Ranking
Show answer and example
Answer: The integer indices choose blocks but do not carry gradients. Gates in attention let prediction loss train selection scores.
Illustrative example: Identical selected blocks with 0.8/0.2 versus 0.5/0.5 produce different outputs and gradients.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Core mechanism
Why can't a cyclic dependency graph have a node-wise parents-first topological order?
Show answer and example
Answer: Nodes in a directed cycle depend on one another, so no linear order satisfies all edges. Identify strongly connected components and separate ordering across components from migration within them. AST context reveals this limit, but it does not invent a cycle-breaking method the paper never defines.
Illustrative example: Example: A calls B and B imports A. Analyze them as a cycle, not as though migrating A first satisfies all parent dependencies.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Implementation details
How should static graphs handle local-variable shadowing and unresolved symbols?
Show answer and example
Answer: Same names can denote different definitions, and local binding hides globals. Do not connect unresolved calls to arbitrary same-name repository nodes. Retain scope and unresolved status, then retrieve only known structure. Local support is bounded Python AST, not all dynamic calls or other paper languages.
Illustrative example: Example: If a function assigns a local object to a global tool's name, later calls use the local binding, not a global node with the same text.
Sections in the Chinese source: 核心公式, 本地复现, 代码映射与边界. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Method comparison
How does three-hop structural context differ from vector retrieval over 800-character chunks?
Show answer and example
Answer: Vector chunks return textually similar fragments. Structural retrieval follows inheritance, calls, and imports, potentially finding related code with different words. Both miss things. Some paper metrics on a single repository decline, so structure does not uniformly improve migrations.
Illustrative example: Example: A utility's name has little lexical overlap with a migration issue, but a call edge still brings it into context.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Basics
Why identify parent dependencies before migrating callers?
Show answer and example
Answer: Parent classes, imports, and called interfaces shape assumptions in dependent code. Migrating resolvable parents first stabilizes context. HCRG extracts AST structure up to three hops. Static graphs cannot cover all dynamic calls, and unresolved symbols should not be matched to arbitrary same-name definitions.
Illustrative example: Example: Migrating a superclass constructor signature first makes it easier to update subclasses consistently.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why preserve both directions for one pair in CPPA?
UniRec: Cross-Stage Multi-Task Fusion with Preference Optimization for Cascaded Recommender Systems
Show answer and example
Answer: Objectives can disagree, and a majority loses strength and minority signal. Both softplus penalties remain, CAPP selects confident directions, and grouped KL limits drift. Downstream supervision is training-only.
Illustrative example: Clicks favor A while purchases favor B; majority voting can hide the purchase signal.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
How keep cross-stage gradient isolation and AGRR evidence separate from pruning?
UniRec: Cross-Stage Multi-Task Fusion with Preference Optimization for Cascaded Recommender Systems
Show answer and example
Answer: Cross-stage alignment uses survivors and freezes the fine-ranker direction. AGRR builds grouped KL targets from unpruned inputs, not centered scores. Document variance and quantile ties.
Illustrative example: Computing evidence after pruning changes the distribution AGRR should see.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does CPPA differ from majority-voting among objectives?
UniRec: Cross-Stage Multi-Task Fusion with Preference Optimization for Cascaded Recommender Systems
Show answer and example
Answer: Opposing evidence contributes to two softplus terms rather than discarding the minority. CAPP chooses confident directed pairs and AGRR controls group drift separately.
Illustrative example: Clicks can prefer A and watch time B; retain both directions.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why keep two separate pre-ranking and ranking towers?
UniRec: Cross-Stage Multi-Task Fusion with Preference Optimization for Cascaded Recommender Systems
Show answer and example
Answer: Their features, candidate scales, and latency differ. Share some embeddings and align downstream preference through training, without exposing expensive online ranking input upstream.
Illustrative example: Pre-ranking learns fine-ranker preference from supervision, not its full feature set.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why train synthetic trigger/exemption rules and threshold one verdict?
SIRF: A Spec-Internalized Risk Foundation Model for Industrial Content Risk Control
Show answer and example
Answer: Long-tail combinations are scarce in logs. Rule graphs, rewrites, and account reasoning teach combinations during continued pretraining; one verdict saves online work. Threshold trades precision and recall. Local interactions are not production capability.
Illustrative example: Frequent trades can trigger concern, but verified merchant status may exempt them.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
Does the local high-precision threshold reproduce full 8B continued pretraining?
SIRF: A Spec-Internalized Risk Foundation Model for Industrial Content Risk Control
Show answer and example
Answer: No. It uses trigger/exemption proxies, not private accounts, EntiGraph/MAGA generation, or large training. Select thresholds on validation and keep held-out reporting separate.
Illustrative example: A small classifier reaching precision target does not reproduce a safety foundation model.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
How does learned rule internalization differ from a runtime rules engine?
SIRF: A Spec-Internalized Risk Foundation Model for Industrial Content Risk Control
Show answer and example
Answer: Learned models can generalize language and context but need not obey every rule exactly. Explicit engines obey coded conditions with limited coverage. Accuracy is not formal proof.
Illustrative example: A model may retain an old policy after the rule changes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why train on both triggers and exemptions in SIRF?
SIRF: A Spec-Internalized Risk Foundation Model for Industrial Content Risk Control
Show answer and example
Answer: Safety decisions depend on exceptions as well as triggers. Synthetic rules and continued pretraining aim to cover rare combinations; thresholds manage precision.
Illustrative example: A keyword need not violate policy in an explicitly exempt explanatory context.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
Why check stability in addition to confidence during TASCO adaptation?
Beyond Confidence: Stability-Aware Test-Time Adaptation for LLM Reasoning
Show answer and example
Answer: Confidence can reinforce wrong answers. Keep backbone fixed, tune a small prefix and penalize unstable predictions under neighborhood perturbations. Stability is unlabeled evidence, not a correctness oracle; no real local LM prefix training.
Illustrative example: If equivalent wording changes the answer, high original confidence alone is a poor signal.
Sections in the Chinese source: 背景与主要改动, 核心公式或操作. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
Does the local TASCO stability penalty train an actual LM prefix?
Beyond Confidence: Stability-Aware Test-Time Adaptation for LLM Reasoning
Show answer and example
Answer: No. It implements random neighborhood perturbation and a stability reference kernel without a frozen LM prefix path. Small-vector tests do not establish test-time gains.
Illustrative example: Stable numeric features do not show a language model's prefix was optimized on tasks.
Sections in the Chinese source: 背景与主要改动, 核心公式或操作. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
How do TASCO prefix updates and best-of-N spend extra compute differently?
Beyond Confidence: Stability-Aware Test-Time Adaptation for LLM Reasoning
Show answer and example
Answer: TASCO optimizes conditioning before a frozen model predicts; best-of-N generates and selects more candidates. Compare optimizer forwards/backwards and generation, not just candidate count.
Illustrative example: Several gradient steps and several independently generated answers are different workloads.
Sections in the Chinese source: 背景与主要改动, 核心公式或操作. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
Does TASCO adapt backbone weights or a lightweight prefix?
Beyond Confidence: Stability-Aware Test-Time Adaptation for LLM Reasoning
Show answer and example
Answer: Freeze the backbone and optimize an added prefix, changing conditioning without retraining the model or selecting only by sampling. It uses unlabeled current-task signals, never test gold.
Illustrative example: Update prefix vectors and reread probabilities while leaving model layers and answer labels untouched.
Sections in the Chinese source: 背景与主要改动, 核心公式或操作. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Core mechanism
Why not suppress the entire erroneous answer in NSD?
Negative Self-Distillation: Learning to Reason by Avoiding Flaws
Show answer and example
Answer: Negative teacher gives a distribution to avoid, with dynamic gates selecting reasoning-critical tokens while preserving useful language/steps. Selection doesn't prove errors and full LM training wasn't reproduced.
Illustrative example: A carry error needn't suppress every connective and correct digit.
Sections in the Chinese source: 背景与主要改动, 核心公式或操作. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Implementation details
What supplies local NSD negative teachers and gates?
Negative Self-Distillation: Learning to Reason by Avoiding Flaws
Show answer and example
Answer: Candidate policies test control flow, not full-parameter LLM. Freeze and align gates to penalized positions; test gold cannot preidentify errors under no-leakage claims.
Illustrative example: Hand-labeled errors are a unit fixture, not autonomous error localization.
Sections in the Chinese source: 背景与主要改动, 核心公式或操作. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Method comparison
How does negative self-distillation differ from lowering every wrong answer token?
Negative Self-Distillation: Learning to Reason by Avoiding Flaws
Show answer and example
Answer: Wrong responses can contain valid facts, grammar and steps. Selective gating limits collateral damage, but isn't a verified step label.
Illustrative example: A wrong result doesn't mean all numbers and connectives should be forgotten.
Sections in the Chinese source: 背景与主要改动, 核心公式或操作. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Basics
Why use self-generated mistakes as negative teaching material in NSD?
Negative Self-Distillation: Learning to Reason by Avoiding Flaws
Show answer and example
Answer: Failures identify directions to avoid. Negative teacher models error distribution and a dynamic gate concentrates suppression on key reasoning while protecting ordinary language.
Illustrative example: A misused proof rule can supply an avoid signal.
Sections in the Chinese source: 背景与主要改动, 核心公式或操作. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Core mechanism
Why can task-related low-rank updates shorten responses without truncation?
LOCUS: Task-Aware Low-Rank Post-Training for Token-Efficient Language Generation
Show answer and example
Answer: Learning operates on parameter directions and aims to preserve task-relevant changes. Truncation deletes content afterward, potentially essential. Validate subspace, correctness and efficiency; proxy SVD isn't LLM evidence.
Illustrative example: Learn to omit redundant checks, not cut the second half of every proof.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Implementation details
Does local LOCUS SVD train a real language model?
LOCUS: Task-Aware Low-Rank Post-Training for Token-Efficient Language Generation
Show answer and example
Answer: No. It projects policy gradients in a feature subspace. Check rank, orthogonality and gradient paths. A random low-rank matrix isn't learned semantic direction.
Illustrative example: Projection success doesn't establish coverage of core reasoning.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Method comparison
How does low-rank learning differ from truncating output?
LOCUS: Task-Aware Low-Rank Post-Training for Token-Efficient Language Generation
Show answer and example
Answer: Train more concise behavior rather than deleting already generated steps. Check correctness alongside length, not shorter alone.
Illustrative example: Omitting repetition and cutting a proof's conclusion both shorten text but differ in quality.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Basics
Why constrain LOCUS updates to task-related low rank?
LOCUS: Task-Aware Low-Rank Post-Training for Token-Efficient Language Generation
Show answer and example
Answer: Compact directions can improve token efficiency while limiting unrelated drift. If subspace misses key skills, performance can suffer.
Illustrative example: Removing redundant language may not require unrestricted changes to all parameters.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Core mechanism
Why mix FKL and RKL per token rather than select one direction per answer?
Show answer and example
Answer: Uncertainty, gaps and reference drift vary by token. Allocate coverage and concentration accordingly. Gate is a supervision heuristic, not truth, and local tests don't show stable superiority.
Illustrative example: Preserve multiple options at a pivotal branch and use concentrated guidance for settled format.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Implementation details
Can local four-signal gates qualify for capability promotion?
Show answer and example
Answer: No. Deterministic mechanism tests don't establish full-model effects or fair compute. Specify direction, signal versions and gradients; registry changes are not training.
Illustrative example: One improved small-candidate score doesn't generalize to open language.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Method comparison
How does per-token FKL/RKL differ from one ratio per reply?
Show answer and example
Answer: Token gates vary coverage vs concentration by position; fixed mixture applies everywhere. Flexibility increases tuning/overfit risk; diagnostics aren't consistent wins.
Illustrative example: A reasoning branch and predictable format token need not receive the same mix.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Basics
Why combine multiple signals for adaptive OPD gating?
Show answer and example
Answer: Entropy, uncertainty, reference drift and teacher/student gap capture different conditions. Combine them to allocate per-token forward/reverse KL; signal utility must be validated.
Illustrative example: Teacher confident but far from reference differs from uncertain but close to student.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Core mechanism
Why evaluate SWRouter context compression separately from model choice?
SWRouter: Similarity-Contractive Window Routing for Multi-Turn Large Language Model Conversations
Show answer and example
Answer: It first contracts conversation windows by adjacent semantic similarity, then routes. Losing a constraint can make even a good model fail; full context also does not guarantee a good route. Separate measurements locate failures.
Illustrative example: If turn two changes the budget and compression drops it, don't blame the routed model alone.
Sections in the Chinese source: 背景与主要改动, 核心公式或操作. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Implementation details
Is local profile-overlap routing the paper's trained contrastive encoder?
SWRouter: Similarity-Contractive Window Routing for Multi-Turn Large Language Model Conversations
Show answer and example
Answer: No. It executes segmentation and profile-overlap routing without contrastive training. Test these as separate local components, not as paper-level learned performance.
Illustrative example: Selecting from keyword overlap shows rule execution, not learned contrastive representations.
Sections in the Chinese source: 复现边界. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Method comparison
How does semantic contraction differ from keeping the last few turns?
SWRouter: Similarity-Contractive Window Routing for Multi-Turn Large Language Model Conversations
Show answer and example
Answer: A recency window assumes newer is better; semantic contraction uses adjacent meaning to segment. Compare context quality separately from routing accuracy.
Illustrative example: An older key turn need not be removed merely because it is old.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Basics
Why contract multi-turn context before routing?
SWRouter: Similarity-Contractive Window Routing for Multi-Turn Large Language Model Conversations
Show answer and example
Answer: Adjacent semantic similarity can identify repetition or unnecessary history. Construct a budgeted window first, then route on that context.
Illustrative example: Consolidate repeated clarifications before deciding what model is needed.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Core mechanism
How does spatial patch reconstruction differ from predicting discrete image codes?
SenseNova-U1.5: Towards Native Unified Visual Intelligence
Show answer and example
Answer: Discrete methods use a tokenizer or VAE to turn images into codes, then predict those codes. Patch reconstruction targets local spatial structure directly and relies less on that intermediate interface. This changes representation and supervision, not proof of zero bottleneck or lower cost.
Illustrative example: One method predicts codebook IDs and the other reconstructs spatial patches. Their error sources differ.
Sections in the Chinese source: 背景与主要改动, 核心公式或操作. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Implementation details
Do the local masked-patch and expert OPD results establish a unified 8B model?
SenseNova-U1.5: Towards Native Unified Visual Intelligence
Show answer and example
Answer: No. Deterministic reference kernels and three-seed state-transition diagnostics cover masked targets and expert fusion, not multistage 8B training. Gating and loss values verify paths, not real understanding or generation benchmarks.
Illustrative example: Correctly summing expert weights in a tensor does not demonstrate usable images or real visual answers from an eight-billion-parameter model.
Sections in the Chinese source: 核心公式或操作, 本地复现, 复现边界. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Method comparison
How does multi-expert OPD differ from a single teacher?
SenseNova-U1.5: Towards Native Unified Visual Intelligence
Show answer and example
Answer: Experts supply different tasks or abilities; the unified model must reconcile signals rather than match one teacher distribution. Coverage can grow, but so can conflicts. More teachers do not guarantee improvement in both understanding and generation.
Illustrative example: A teacher emphasizing object relations and another emphasizing image reconstruction do not simply repeat the same target.
Sections in the Chinese source: 背景与主要改动, 本地复现. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Basics
Do visual understanding and generation in SenseNova-U1.5 have the same input–output direction?
SenseNova-U1.5: Towards Native Unified Visual Intelligence
Show answer and example
Answer: No. Understanding conditions on vision to answer or judge, while generation produces visual content from conditions. A unified model needs both signals. Spatial patch reconstruction concerns generation representations; expert OPD concerns post-training supervision. One operator does not establish both capabilities.
Illustrative example: Answering what is in an image and creating an image on request are different acceptance tests.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Core mechanism
Why can repeated data affect MoE and dense models differently at equal token counts?
Data Scarcity and Model Sparsity: Mixtures-of-Experts Overfit More to Repeated Data
Show answer and example
Answer: Token count does not equal unique information. Routed experts may memorize repeated samples faster. Study stronger masking/dropout, and use held-out data; decreasing train loss is insufficient.
Illustrative example: Ten passes over one corpus differ from ten distinct corpora despite equal token budgets.
Sections in the Chinese source: 背景与主要改动, 核心公式或操作. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Implementation details
How should a local repeat-aware schedule be accepted?
Data Scarcity and Model Sparsity: Mixtures-of-Experts Overfit More to Repeated Data
Show answer and example
Answer: Check that repeat and sparsity actually change masking/dropout according to the definition, and record applied settings. State-transition fixtures are not large pretraining or generalization evidence.
Illustrative example: Higher repetition should alter effective regularization, not just a config name.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Method comparison
Is matching steps enough to compare repeated-data overfitting?
Data Scarcity and Model Sparsity: Mixtures-of-Experts Overfit More to Repeated Data
Show answer and example
Answer: Match or report total tokens, unique data, repetition, and sparsity. The work studies memorization under these conditions, not universal MoE inferiority.
Illustrative example: If one arm sees more repeats, architecture alone cannot explain its generalization gap.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Basics
Why treat repetition as an independent variable?
Data Scarcity and Model Sparsity: Mixtures-of-Experts Overfit More to Repeated Data
Show answer and example
Answer: Same tokens can contain different unique information. Repetition can worsen expert memorization; study its interaction with sparsity and regularizers.
Illustrative example: Ten reads of 10,000 examples differ from one read of 100,000 examples.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Core mechanism
Why not share one global quantization range and rotation across multimodal KV?
OmniKVQuant: KV Cache Quantization for Omni-LLMs
Show answer and example
Answer: Keys may drift over time so a global range loses local precision. Value geometry also differs by modality. Short-window ranges handle temporal shifts and modality-specific rotations handle value geometry; one static rule need not control both errors.
Illustrative example: Small audio keys can lose precision when later video keys set the global range; value structure still needs separate checks.
Sections in the Chinese source: 背景与主要改动, 核心公式或操作. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Implementation details
Does the local 2-bit NumPy reference demonstrate a faster GPU KV kernel?
OmniKVQuant: KV Cache Quantization for Omni-LLMs
Show answer and example
Answer: No. It checks window quantization and modality-specific transforms. Actual Triton or other kernels need real A100/A30 validation. Packing, unpacking, memory traffic, and fusion affect latency beyond bit width.
Illustrative example: Theoretically two-bit values stored and decoded in floating arrays do not yet deliver real two-bit VRAM or bandwidth savings.
Sections in the Chinese source: 核心公式或操作, 本地复现, 复现边界. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Method comparison
How does modality-specific rotation differ from using one matrix for all values?
OmniKVQuant: KV Cache Quantization for Omni-LLMs
Show answer and example
Answer: A shared rotation assumes one coordinate transform suits all modalities. Separate rotations allow distinct coordinate systems for audio, vision, and text. Accuracy still depends on calibration and actual distributions.
Illustrative example: Image and audio values may have different high-variance directions; one rotation might suit only one.
Sections in the Chinese source: 背景与主要改动, 核心公式或操作. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Basics
Does OmniKVQuant compress weights or the current context's KV cache?
OmniKVQuant: KV Cache Quantization for Omni-LLMs
Show answer and example
Answer: It targets key/value states stored during inference, not fixed network parameters. Cache grows with context and contains different modality representations; quantized weights do not remove it.
Illustrative example: The same model has identical weights for short text and long audiovisual context, but different KV storage.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Core mechanism
How does Musec spectral clipping differ from shrinking all gradients?
Musec: MomentUm SpEctral Clipping for Stable Muon-type Training
Show answer and example
Answer: Smoothly constrain large momentum singular values without uniformly shrinking every direction. This changes update geometry, not proof of stability or final quality.
Illustrative example: Control one abnormally large direction without multiplying every normal direction by 0.1.
Sections in the Chinese source: 背景与主要改动, 核心公式或操作. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Implementation details
Does local full-SVD soft clipping establish large-model throughput?
Musec: MomentUm SpEctral Clipping for Stable Muon-type Training
Show answer and example
Answer: No. It validates decomposition, smooth mapping, and reconstruction, not large training or optimized kernels. Verify numerical correctness and runtime separately.
Illustrative example: Small-matrix parity does not establish cost per step on a large model.
Sections in the Chinese source: 本地复现, 复现边界. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Method comparison
What differs between smooth spectral clipping and a hard threshold?
Musec: MomentUm SpEctral Clipping for Stable Muon-type Training
Show answer and example
Answer: Hard decisions can jump at the boundary; smooth mapping responds continuously. Specify the exact rule and budget rather than just claiming stability.
Illustrative example: Slightly crossing a threshold need not abruptly change update magnitude.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Basics
What instability does Musec target?
Musec: MomentUm SpEctral Clipping for Stable Muon-type Training
Show answer and example
Answer: Smoothly clip excessive momentum singular directions so they do not dominate Muon-type updates. Controlling spectrum alone does not guarantee better loss.
Illustrative example: Limit a few huge singular values without equally scaling normal ones.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Core mechanism
Why add a visual-need router after building caption memory?
Show answer and example
Answer: Text remembers broad events but may omit details needed later. The router decides whether text evidence suffices and spends more vision budget only when needed. This controls evidence-acquisition cost; captions are not lossless and a routing decision is not an answer.
Illustrative example: “Someone picks up a box” can answer whether an object was taken, but reading its serial number may require looking at frames.
Sections in the Chinese source: 背景与主要改动, 核心公式或操作. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Implementation details
Do three-seed local caption/query gates validate the learned visual-need router?
Show answer and example
Answer: No. Reference state transitions check gating, top-K frame selection, and budget; there is no long-video VLM or trained semantic router. This does not prove the system can tell which questions text answers or reproduce frame savings.
Illustrative example: A rule can select at most three frames without knowing whether any contains the answer.
Sections in the Chinese source: 核心公式或操作, 本地复现, 复现边界. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Method comparison
What is the trade-off versus uniformly resampling the video for every question?
Show answer and example
Answer: Uniform sampling repeatedly pays vision cost. On-demand access reuses a caption and focuses on uncertain frames, but a missed detail or routing error can hide key evidence. Compare real quality on identical questions and budgets.
Illustrative example: Ten event-order queries may share memory; a brief action in the last query still needs the right frame.
Sections in the Chinese source: 背景与主要改动, 核心公式或操作. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Basics
Does caption-once mean writing a new video caption for each question?
Show answer and example
Answer: No. Build one reusable text memory, then route and budget additional vision separately for each question. Count initial captioning and per-query frame access separately; memory reuse does not mean all queries avoid visual compute.
Illustrative example: Five questions can share one caption, and only the question needing additional detail may require frames.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Core mechanism
Why can starting subsequent problems from an accepted program accumulate optimization knowledge better than reprompting from scratch?
MAPLE: Memory-Augmented Planning with Language and Evolution
Show answer and example
Answer: Program state preserves verified executable structure and trade-offs. The next task can modify it rather than reconstruct details from language. Acceptance prevents unexecuted ideas from overwriting a reliable parent. Cross-problem evolution still needs to account for task differences: a specialized program for one instance is not universally best.
Illustrative example: If the previous problem already has reliable parsing, change its objective for the next task rather than regenerate a blank script and repeat format mistakes.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Implementation details
How does local validation of nonempty named-operation plans differ from the authors' solver?
MAPLE: Memory-Augmented Planning with Language and Evolution
Show answer and example
Answer: The local path accepts public plans made of nonempty named operations and checks reuse. It has no author solver, trained model, or external tools. Structural execution contracts can be tested, but perfect mini-suite scores are not paper-level optimization or general program evolution. A missing solver cannot be substituted with a registry name.
Illustrative example: Nodes named 'search' and 'verify' pass the nonempty structural check, but do not demonstrate that a real search algorithm outperforms its predecessor.
Sections in the Chinese source: 本地复现与边界. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Method comparison
How does accepting an executable program differ from accepting an optimization description that sounds better?
MAPLE: Memory-Augmented Planning with Language and Evolution
Show answer and example
Answer: A description conveys intent. A program must run and pass the relevant comparison before becoming evolvable state. Executability does not imply improvement: validity and benefit are separate gates. A one-time gain does not generalize to all future tasks, and failed proposals should not be inherited unconditionally.
Illustrative example: 'Reduce redundant search' is advice. Accept the optimization only after it changes the call logic and verifies that required evidence was not missed.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Basics
Why does MAPLE aim to retain optimization knowledge across consecutive solves?
MAPLE: Memory-Augmented Planning with Language and Evolution
Show answer and example
Answer: Related tasks often share structure. Starting each proposal from scratch makes verified program improvements hard to accumulate. MAPLE uses executable solutions as persistent research state and explores subsequent problems from accepted programs. Inheritance can carry stale assumptions, so verify the new constraints rather than treating executability or historical success as proof of current optimality.
Illustrative example: Reuse a scheduling program that already resolves resource conflicts, but implement and test the new time-window constraint.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Core mechanism
Why aggregate similar failures across tasks to diagnose a systemic harness defect?
Ecdysis: Efficient and Effective Training of Runtime Harnesses for LLM Agents
Show answer and example
Answer: A single failure may be due to randomness or a particular prompt. A recurring pattern more strongly suggests a shared execution interface problem. Multi-role diagnosis turns the evidence into a testable repair specification instead of changing global behavior after each failure. Correlation is not sufficient causal proof; compare affected behaviors after repair.
Illustrative example: Many tasks losing error codes when logs are truncated gives stronger reason to inspect the output interface than one bad guess. Target truncation rather than generally replacing the model.
Sections in the Chinese source: 背景与主要改动, 核心公式或操作. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Implementation details
Does triggering the local FDCR repair gate mean a harness patch was generated or executed?
Ecdysis: Efficient and Effective Training of Runtime Harnesses for LLM Agents
Show answer and example
Answer: No. The local path aggregates signatures from public observations and triggers a bounded repair gate. It does not generate or execute code or run paper-scale training. A trigger shows the statistics satisfy a condition, not correct diagnosis or actual repair. Three-seed counts cannot be converted into paper speed or accuracy.
Illustrative example: A 'needs repair' state after three similar failures is a scheduling signal. Without a patch and rerun, the defect is not resolved.
Sections in the Chinese source: 核心公式或操作, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Method comparison
Where do a repair specification and direct execution of a harness patch sit in the improvement chain?
Ecdysis: Efficient and Effective Training of Runtime Harnesses for LLM Agents
Show answer and example
Answer: FDCR's multi-role diagnosis organizes failure evidence into a discussable repair specification describing what to change and why. Execution still requires a concrete patch, runtime integration, and verification. A diagnosis is not a changed system; gate triggers or counts of repair suggestions do not indicate task-success gains. This boundary is especially important for the local reference.
Illustrative example: A specification says 'preserve tool error fields.' Check the patch and execution logs separately to see whether the code actually changed and worked in later tasks.
Sections in the Chinese source: 背景与主要改动, 核心公式或操作. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Basics
Why does Ecdysis ask whether failures are occasional model behavior or shared runtime defects?
Ecdysis: Efficient and Effective Training of Runtime Harnesses for LLM Agents
Show answer and example
Answer: Failures across tasks can be individual misunderstandings, or recurrent effects of tool wrapping, context, and recovery. Ecdysis looks across executions for repairable harness defects so changes target shared causes. Recurrence is a diagnostic clue, not a completed fix without execution and verification.
Illustrative example: If several tasks lose evidence because tool output is truncated, suspect a common interface. A single task that misreads content does not justify immediately changing every workflow.
Sections in the Chinese source: 背景与主要改动, 核心公式或操作. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Core mechanism
Why use a budgeted contextual bandit for dynamic skill candidates rather than ranking historical best scores alone?
COBRA-Skills: Contextual Bandit-Guided Evolution for Agent Skill Optimization
Show answer and example
Answer: Skill payoff depends on the current task, and new or revised candidates lack history. A contextual bandit balances exploiting known gains and exploring informative candidates under an evaluation budget, then evolves from execution feedback. Historical maxima can favor early lucky skills and ignore context and uncertainty about untested candidates.
Illustrative example: If an old skill excels at files but new tasks are web-based, giving new candidates a few trials may be more informative than repeatedly testing the old champion.
Sections in the Chinese source: 背景与主要改动, 核心公式或操作. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Implementation details
Does local contextual-UCB allocation cover actual LLM skill evolution?
COBRA-Skills: Contextual Bandit-Guided Evolution for Agent Skill Optimization
Show answer and example
Answer: No. It executes a reference allocator and feedback admission gate, with no external LLM or six environments. Candidate indices, confidence terms, and budget can be checked independently, but actual skill-text generation and task execution did not occur. Fixture feedback is not actual cost savings or capability gain.
Illustrative example: Selecting a numeric arm after its UCB grows checks the bandit formula. It does not demonstrate that the corresponding natural-language skill works on web tasks.
Sections in the Chinese source: 核心公式或操作, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Method comparison
Why separate proposing a skill variant from measuring its execution reward?
COBRA-Skills: Contextual Bandit-Guided Evolution for Agent Skill Optimization
Show answer and example
Answer: A proposal uses existing experience to describe potentially better behavior but does not establish usefulness on this task. Only execution supplies feedback for budget allocation and evolution. The space may grow dynamically, but membership does not imply positive reward. COBRA-Skills bandit selection must distinguish measured payoff from uncertainty rather than fill missing results with proposal self-assessment.
Illustrative example: A more detailed skill can be scheduled for a trial. Only actual execution shows whether it helps; 'expected improvement' is not a successful sample.
Sections in the Chinese source: 背景与主要改动, 核心公式或操作. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Basics
Why can't COBRA-Skills give each candidate the same number of trials as the skill pool grows?
COBRA-Skills: Contextual Bandit-Guided Evolution for Agent Skill Optimization
Show answer and example
Answer: Evaluating skills consumes execution budget while the number and quality of candidates change. Equal allocation can waste trials on clearly useless or duplicate entries. COBRA-Skills combines current context with past feedback to direct limited trials. Selectors may only use available evidence, not presume effects of unexecuted candidates.
Illustrative example: Ten similar skills have repeatedly failed and a newly generated one may deserve limited exploration. Running every candidate a hundred times is not feasible on a tight budget.
Sections in the Chinese source: 背景与主要改动, 核心公式或操作. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Core mechanism
Why compare post-trained teacher to its own reference rather than absolute teacher–student scores?
CompassOPD: Cross-Family On-Policy Distillation via Within-Family Likelihood Shifts
Show answer and example
Answer: Calibration and sharpness vary by family. Relative likelihood shifts and within-family centering extract direction, with a student reference anchor. Teacher bias can remain.
Illustrative example: Two teachers may both increasingly favor the right step despite very different initial distributions.
Sections in the Chinese source: 背景与主要改动, 核心机制. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Implementation details
What comparable sets should Compass center over?
CompassOPD: Cross-Family On-Policy Distillation via Within-Family Likelihood Shifts
Show answer and example
Answer: Use consistent candidates/weights for specified family changes and a separate student reference. Don't center arbitrarily across questions. Local audit magnitudes aren't training success.
Illustrative example: Mismatched option order makes subtraction meaningless.
Sections in the Chinese source: 背景与主要改动, 核心机制. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Method comparison
How does Compass trade off against averaging cross-family probabilities?
CompassOPD: Cross-Family On-Policy Distillation via Within-Family Likelihood Shifts
Show answer and example
Answer: Relative directions mitigate scale effects but discard some global information. Direct mixtures retain absolute posteriors. Compare under equal protocol; no universal win.
Illustrative example: Two moves toward the correct option shouldn't be obscured by native temperature differences.
Sections in the Chinese source: 背景与主要改动, 核心机制. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Basics
Why does CompassOPD need each teacher's own reference?
CompassOPD: Cross-Family On-Policy Distillation via Within-Family Likelihood Shifts
Show answer and example
Answer: Absolute probability scales differ across families. Relative change captures post-training direction, and within-family centering reduces global bias before using the student anchor. It isn't correctness proof.
Illustrative example: A naturally sharp teacher shouldn't look more learned just because probabilities are larger.
Sections in the Chinese source: 背景与主要改动, 核心机制. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
Why train on token IDs actually sampled in rollouts rather than retokenizing complete interaction text?
T1: Terminal Agent Reinforcement Learning for Long-Horizon Tasks
Show answer and example
Answer: Joining tool observations and turns before tokenization can alter boundaries or token IDs, so policy probabilities no longer correspond to sampled actions. TITO retains sampled IDs, boundary repair handles drift, and MoE route replay addresses expert-path differences. Alignment is a training contract, not evidence that terminal tasks were learned.
Illustrative example: Concatenating a command and an observation can create different subtokens or include non-action content. Keep sampled IDs to align training with the original action.
Sections in the Chinese source: 背景与主要改动, 核心公式或操作. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
Does a local exact-token/route audit cover training a 122B MoE?
T1: Terminal Agent Reinforcement Learning for Long-Horizon Tasks
Show answer and example
Answer: No. It checks sampled sequences and route replay contracts, without a 122B model, cloud sandboxes exceeding 300 turns, or full terminal RL. Log consistency does not demonstrate a smaller training–inference gap. Paper scores and gap measurements belong to the authors' real system, not fixtures with matching fields.
Illustrative example: A fixture route array that replays element by element confirms indexing, not actual MoE dispatch or backpropagation.
Sections in the Chinese source: 核心公式或操作, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
How does MoE route replay differ from replaying generated tokens alone?
T1: Terminal Agent Reinforcement Learning for Long-Horizon Tasks
Show answer and example
Answer: Expert routes may differ for identical tokens because of implementation or state. R3 saves and replays routing choices to narrow execution-path mismatch between training and inference. Token equality is necessary but insufficient; route equality also does not guarantee task gains without real model and log-probability comparisons.
Illustrative example: If a token originally selects experts A and B but training recomputes A and C, outputs and gradients can change despite identical token IDs.
Sections in the Chinese source: 背景与主要改动, 核心公式或操作. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why must inference trajectories match optimizer examples when training large terminal agents?
T1: Terminal Agent Reinforcement Learning for Long-Horizon Tasks
Show answer and example
Answer: Terminal interactions include generated tokens, observations, and potentially changing execution paths. Reconstructed samples can optimize behavior that was never sampled. T1 keeps long rollouts traceably aligned with updates and supports scale. Record consistency is necessary for correct training, but does not establish capability or final gains.
Illustrative example: If tool output sits between model turns and concatenation changes token boundaries, computed probabilities may no longer represent the original action.
Sections in the Chinese source: 背景与主要改动, 核心公式或操作. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why might more synthetic skill data hurt retrieval on real tasks?
When Synthetic Data Hurts: On Catastrophic Forgetting in Skill Retrieval for LLM Agents
Show answer and example
Answer: Synthetic language and distributions can differ from real requests, shifting embeddings and routing boundaries toward synthetic patterns. Replay preserves real task coverage; anchoring or distillation from an old model limits drift. These protect transfer and retention, not a guarantee that more synthetic data helps or forgetting disappears.
Illustrative example: Example: Synthetic requests always spell out tool names, while real users use abbreviations. Training only on synthetic wording may miss real requests; replaying abbreviations preserves old routing.
Sections in the Chinese source: 背景与主要改动, 核心公式或操作. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
Does passing local anchor, conflict penalty, and replay checks include training the 0.6B retriever?
When Synthetic Data Hurts: On Catastrophic Forgetting in Skill Retrieval for LLM Agents
Show answer and example
Answer: No. The local reference runs route anchors and associated constraints without training the 0.6B model over more than thirty thousand skills. It checks replay participation, anchor-loss direction, and conflict handling. Synthetic state changes do not establish real or OOD retrieval retention.
Illustrative example: Example: Pulling one vector toward its old anchor verifies the gradient, not correct skill retrieval for unseen wording.
Sections in the Chinese source: 核心公式或操作, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
What do real-data replay and embedding anchors or LwF constrain differently?
When Synthetic Data Hurts: On Catastrophic Forgetting in Skill Retrieval for LLM Agents
Show answer and example
Answer: Replay exposes optimization to old tasks; anchors or Learning without Forgetting limit representation or output drift. One provides data coverage and the other constrains behavioral changes induced by updates. They are complementary, not redundant. Too much weight can inhibit new skill learning, so validate the trade-off.
Illustrative example: Example: Replay old colloquial requests and separately constrain their embeddings not to move far. That protects old retrieval through both examples and representation.
Sections in the Chinese source: 背景与主要改动, 核心公式或操作. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Why evaluate both new skills and existing user needs when training skill retrieval?
When Synthetic Data Hurts: On Catastrophic Forgetting in Skill Retrieval for LLM Agents
Show answer and example
Answer: New data adapts the retriever to a larger skill catalog, but service still receives familiar real and out-of-domain queries. The goal is retaining old routing while learning new material, not only minimizing loss on new data. Improvements on new tasks do not erase the cost of forgetting existing usage.
Illustrative example: Example: A model works on synthetic examples with canonical function names, but users still describe their needs colloquially. Those queries are not obsolete.
Sections in the Chinese source: 背景与主要改动, 核心公式或操作. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Core mechanism
Why can an evidential query graph reveal a gap after searching many web pages?
SearchAtlas: Analyzing Agentic Search Strategies via Evidential Query Graphs
Show answer and example
Answer: Page count is not constraint coverage. Directed query–evidence–answer relations must show what supports each condition and whether the conclusion crosses an unverified edge. Graph auditing separates repeated evidence from a genuinely completed reasoning chain. A passing graph interface does not establish reliable evidence collection by the search model.
Illustrative example: Example: Ten pages give a product's price, but none confirms regional availability. A claim that it is sold locally still lacks a support edge.
Sections in the Chinese source: 背景与主要改动, 核心公式或操作. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Implementation details
Does local validation of evidence-propagation edges include the paper's search agents and automatic parser?
SearchAtlas: Analyzing Agentic Search Strategies via Evidential Query Graphs
Show answer and example
Answer: No. The local run builds and checks edges on public fixtures and does not run the paper's five search agents. The paper's edge F1 measures agreement between automatic graph parsing and human annotations, not local QA accuracy. Executing the interface does not establish evidence extraction or fact judgment on real pages.
Illustrative example: Example: Checking connectivity after being given an edge from Evidence A to Constraint B does not prove the model can extract that relationship from arbitrary web pages.
Sections in the Chinese source: 核心公式或操作, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Method comparison
Why are search diversity and constraint coverage different metrics?
SearchAtlas: Analyzing Agentic Search Strategies via Evidential Query Graphs
Show answer and example
Answer: Diversity measures differences among queries or sources; coverage measures whether evidence supports the requested conditions. Distinct sites can repeat one fact, and a few strong sources can cover several needs. SearchAtlas audits the support structure rather than treating domain counts as evidence chains.
Illustrative example: Example: Five sites can repeat the same product specifications without covering returns. One official policy page can fill that gap.
Sections in the Chinese source: 背景与主要改动, 核心公式或操作. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Basics
Why are edges in a query–evidence–answer graph more explanatory than a list of pages?
SearchAtlas: Analyzing Agentic Search Strategies via Evidential Query Graphs
Show answer and example
Answer: Edges show what evidence a query found and which constraint or answer span it supports. This traces how information reaches the conclusion. A link list records visits without establishing support. Graph extraction itself can be wrong, so an edge is not definitive proof of a fact.
Illustrative example: Example: If all sources discuss price, the graph can show no incoming edge for local availability, which is harder to spot in a list of ten links.
Sections in the Chinese source: 背景与主要改动, 核心公式或操作. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why should a memory curator probe the environment with least-privilege read-only tools before writing?
Grounding Agent Memory: Environment-Probing Curation for Enterprise Agents
Show answer and example
Answer: Conversation claims may be stale or environment-specific. Probing ties candidate memory to current observable facts and scope. Read-only least privilege avoids changing the observed system. A verified claim holds under those conditions, not permanently or everywhere.
Illustrative example: Example: A claim that the server runs on port eight thousand can be checked in read-only configuration and scoped to that environment. Do not edit the config to make it true.
Sections in the Chinese source: 背景与主要改动, 核心公式或操作. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
Why must a curator not turn a least-privilege read-only tool into a repair action?
Grounding Agent Memory: Environment-Probing Curation for Enterprise Agents
Show answer and example
Answer: The curator verifies candidate memory. Writing changes the state it is trying to observe and introduces side effects. Local tests cover repeated probing, conflict rejection, and admission, not Copilot SDK integration. Passing call contracts does not reproduce enterprise agent costs or outcomes.
Illustrative example: Example: If a value differs, update the memory's evidence rather than editing live configuration back to the old value and calling the old memory verified.
Sections in the Chinese source: 核心公式或操作, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
How does admission by environment probing differ from direct LLM summarization of experience?
Grounding Agent Memory: Environment-Probing Curation for Enterprise Agents
Show answer and example
Answer: Summarization relies on trace text and can turn guesses into facts. Independent probes can reject contradictions and narrow scope, but add query and maintenance cost and can verify only observable conditions. A verification step does not make every enterprise state reliable.
Illustrative example: Example: A missing path in one repository should not become a claim about all projects. A probe can scope it to the current repository and revision.
Sections in the Chinese source: 背景与主要改动, 核心公式或操作. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Why can't an enterprise agent treat a guess after one execution as long-term knowledge?
Grounding Agent Memory: Environment-Probing Curation for Enterprise Agents
Show answer and example
Answer: Permissions, configuration, and interfaces change, and one failure may be temporary. Grounded Memory requires traceable environmental evidence and a scope before persisting experience. Recording what happened is only part of knowing where and when it holds.
Illustrative example: Example: A repository access failure today does not mean the service is permanently unavailable. A durable conclusion needs clearer state evidence.
Sections in the Chinese source: 背景与主要改动, 核心公式或操作. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
What does a fixed remote sketch bound, and what still scales?
SequenceO1: End-to-End Ultra-Long (100K) Sequence Modeling in Recommendation with Low-Rank Caching
Show answer and example
Answer: Candidate–sketch matching uses fixed prototypes; recent behavior remains detailed and STCA connects timescales. Reading, building, and updating the sketch still cost work. The local proxy is not the industrial path.
Illustrative example: Matching dozens of prototypes is bounded, but constructing them from 100,000 events is not free.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Is the local 0.65/0.35 blend the paper's 100K-history formula?
SequenceO1: End-to-End Ultra-Long (100K) Sequence Modeling in Recommendation with Low-Rank Caching
Show answer and example
Answer: No. It is a public scoring proxy; no production STCA or long-history service. Fix normalization and candidate masks and do not tune blend on test.
Illustrative example: Small-catalog score gains do not show end-to-end latency with 100,000 events.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does sketch plus recent differ from recent-only truncation?
SequenceO1: End-to-End Ultra-Long (100K) Sequence Modeling in Recommendation with Low-Rank Caching
Show answer and example
Answer: Sketch preserves compressed long-term patterns, whereas truncation drops remote history. Prototypes can still merge different interests.
Illustrative example: An older sports preference can survive a temporary travel streak.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Is a SequenceO1 sketch original distant history?
SequenceO1: End-to-End Ultra-Long (100K) Sequence Modeling in Recommendation with Low-Rank Caching
Show answer and example
Answer: No. A fixed-budget prototype representation summarizes remote behavior while recent history remains finer. It is not event-by-event recoverable.
Illustrative example: Thousands of events can form a few prototypes without preserving exact times.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Core mechanism
Why does BAFF check both rank and bid-price differences?
BAFF: Bid-Aware Filter Family for Mitigating Training Data Interference in RTB A/B Tests
Show answer and example
Answer: Similar ranks can still imply different win probabilities if bids differ; similar bids can still produce different exposure when ranks move. The two conditions remove A logs likely to affect policy B's decisions and reduce training interference. Retained samples are more transferable under these conditions, not evidence of a truly interference-free market.
Illustrative example: A similarly ranked ad may win because A bids much higher. Conversely, equal bids do not make logs interchangeable if a rank crosses the display boundary.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Implementation details
Why does BAFF's bid-price distance count only the positive excess of A over B?
BAFF: Bid-Aware Filter Family for Mitigating Training Data Interference in RTB A/B Tests
Show answer and example
Answer: The paper uses `max(bpA−bpB,0)/bpA`, which measures upward price movement in that direction, not a symmetric absolute gap. Rank distance is the position in B divided by K. Keep the direction, normalization, and two separate thresholds.
Illustrative example: A bid of 10 versus B's 8 yields 20%; A's 8 versus B's 10 does not yield the same positive penalty. An absolute difference would change the filter.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Method comparison
How does BAFF's filtering differ from uniformly downweighting all A logs?
BAFF: Bid-Aware Filter Family for Mitigating Training Data Interference in RTB A/B Tests
Show answer and example
Answer: Uniform downweighting preserves all samples, including opportunities severely affected by A. The two-condition filter rejects incompatible logs based on rank position under B and the bid difference. This can reduce interference at the cost of sample count; stricter thresholds are not always better and do not ensure unbiased survivors.
Illustrative example: Halving the weight of a log with similar prices but very different ranks still uses it. Filtering rejects it, but an overly strict filter can leave too little data and raise variance.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Basics
Why does treatment interference arise in the RTB A/B logs studied by BAFF?
BAFF: Bid-Aware Filter Family for Mitigating Training Data Interference in RTB A/B Tests
Show answer and example
Answer: Policy choices affect bidding opportunities and prices. An ad decision in one arm may change the competition the other policy actually faces, so A's logs are not automatically examples of what B would observe. Filtering approximately interference-free cases limits transfer; it does not make the arm labels interchangeable.
Illustrative example: When A raises an ad's bid, it can alter the competition. B may not have won the same opportunity at the same price, so treating the record as ordinary B training data imports A's policy effect.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Core mechanism
Why isn't transport equivalent to boosting every high-probability teacher token?
Distillation as Probability Transport: Routed On-Policy Distillation
Show answer and example
Answer: Select sources/destinations by student over/underestimation relative to teacher, not absolute teacher mass. This traces corrections and conserves probability. Interpretability doesn't prove teacher correctness; control support and evaluation.
Illustrative example: An already popular word in both models may need no boost; a teacher-supported overlooked word can receive mass.
Sections in the Chinese source: 背景与主要改动, 核心机制. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Implementation details
What invariants should RouteOPD transport check?
Distillation as Probability Transport: Routed On-Policy Distillation
Show answer and example
Answer: Never remove more mass than available; additions equal removals; final probabilities nonnegative and normalized. No transport at zero gap. A kernel doesn't reproduce teacher-model training.
Illustrative example: Removing one unit and adding only half breaks normalization.
Sections in the Chinese source: 背景与主要改动, 核心机制. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Method comparison
How does transporting mass differ from replacing the student distribution with teacher probabilities?
Distillation as Probability Transport: Routed On-Policy Distillation
Show answer and example
Answer: Transport addresses selected discrepancies, not every coordinate. Require nonnegativity, conservation and defined source/destination; ordinary softmax isn't routing.
Illustrative example: Unrouted candidates can remain unchanged, unlike full replacement.
Sections in the Chinese source: 背景与主要改动, 核心机制. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Basics
Why distinguish source and destination tokens in RouteOPD?
Distillation as Probability Transport: Routed On-Policy Distillation
Show answer and example
Answer: Student-overestimated items have excess mass, underestimated items need mass. Explicit transport identifies where corrections come from and go instead of indiscriminate matching.
Illustrative example: Move mass from overpreferred wording to a teacher-supported key digit.
Sections in the Chinese source: 背景与主要改动, 核心机制. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
Why use answer-independent probe consistency and not reinforce the majority at low agreement?
Show answer and example
Answer: Self-labeling can repeat errors. Output-free probes estimate consistency; high/low branches differ. Low agreement constrains lower ranks while entropy preserves exploration. Consistency is not supervised correctness.
Illustrative example: Conflicting probes justify suppressing weak options without blindly rewarding the highest vote count.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
May Probe-ERPO's negative rank mask read the answer key?
Show answer and example
Answer: No. Use allowed probes and candidate rankings only; gold appears in independent scoring. A fixed mask validates the update path, not real probe discrimination.
Illustrative example: A fixture can specify which positions to suppress without teaching automatic error detection.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
Why aren't high-PCR reinforcement and low-PCR negative ranking ordinary pseudo-labeling?
Show answer and example
Answer: Low consistency supplies no reliable positive, so constrain poor candidates rather than canonize the majority. High agreement is only a clue; entropy control and independent evaluation still matter.
Illustrative example: When candidates conflict, avoid weak ones without declaring one correct.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
Why does Probe-ERPO use probes independent of final answers?
Show answer and example
Answer: Gold is unavailable during test-time adaptation. Using it would defeat unlabeled adaptation. PCR uses probe consistency to decide reinforcement vs negative constraints.
Illustrative example: Check agreement from intermediate states instead of matching to gold.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Core mechanism
Why transfer weak-model direction without asking the strong model to imitate answers?
Eliciting Weak-to-Strong Generalization with On-Policy Reverse Distillation
Show answer and example
Answer: Extract centered normalized weak-after/base difference and boost aligned strong-verifier components. The transfer object is direction, not content. Utility requires alignment and candidate diagnostics don't prove broad gains.
Illustrative example: A stronger model can learn to check units without copying an incorrect weak numeric result.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Implementation details
How handle zero change and gradient direction in OPRD?
Eliciting Weak-to-Strong Generalization with On-Policy Reverse Distillation
Show answer and example
Answer: Center and unit-normalize deltas; treat zero norm as no direction. Add projection to original gradient, not replace it. Local candidates only map the proxy to parameters.
Illustrative example: Identical weak checkpoints should not generate a divide-by-zero teaching direction.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Method comparison
How does OPRD gradient augmentation differ from full weak-teacher distillation?
Eliciting Weak-to-Strong Generalization with On-Policy Reverse Distillation
Show answer and example
Answer: Keep strong verifier gradient and amplify its component along weak learning direction, instead of fitting weak distribution. An irrelevant direction may not help.
Illustrative example: Strong model still follows its reward with directional modulation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Basics
What information transfers from weak to strong in OPRD?
Eliciting Weak-to-Strong Generalization with On-Policy Reverse Distillation
Show answer and example
Answer: Weak post-training logit change relative to base augments strong verifier gradient. Weak answers needn't outperform strong ones; a useful update direction may transfer.
Illustrative example: A learned unit-check tendency can help without copying weak arithmetic.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Core mechanism
Why localize experience as a graph edit before a held-out gate?
Procedural Graphs: Self-Evolving Execution Structures for LLM Agents
Show answer and example
Answer: Procedural relations affect multiple paths, and overwriting them with whole episodes can contaminate unrelated skills. Local edits identify the affected node or relation; held-out verification tests whether the change goes beyond incidental success in the originating trace. A pass supports only its evaluation scope, not global reliability inferred from coherent training traces.
Illustrative example: Add only a recovery edge to refresh a token after authentication failure. Check independent tasks for redundant authentication rather than rewriting the whole upload graph.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Implementation details
Does preserving the first and last nodes locally constitute full held-out capability verification?
Procedural Graphs: Self-Evolving Execution Structures for LLM Agents
Show answer and example
Answer: No. The local deterministic edit gate requires existing routes to retain endpoints and tests acceptance and rejection without real tools. It checks a structural condition but cannot establish semantic correctness of middle steps or generalization of the edited graph. The policy must not see hidden plans or answers.
Illustrative example: A graph can retain start and end while deleting required authentication and still pass structurally. That contract cannot replace real task-success evaluation.
Sections in the Chinese source: 本地复现与边界. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Method comparison
What trade-offs arise from a local graph edit versus rewriting the entire procedure graph?
Procedural Graphs: Self-Evolving Execution Structures for LLM Agents
Show answer and example
Answer: Local changes reduce unrelated effects and ease attribution and rollback. Whole-graph rewrites allow more freedom but can break old interfaces and valid paths. Locality does not guarantee safety: one central node can affect many downstream procedures. Check independent behavior, not just graph format.
Illustrative example: Replacing a query node can change all downstream inputs. Despite editing one node, check whether its dependent routes still work.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Basics
What does a procedure–relation–procedure graph add beyond storing skill snippets?
Procedural Graphs: Self-Evolving Execution Structures for LLM Agents
Show answer and example
Answer: A graph explicitly represents dependencies and compositions, so new experience can change a local execution structure rather than add similar prose. Correct relations matter for downstream paths. Validate edits on an independent holdout before permanent write-back, not just because a trace succeeded once.
Illustrative example: 'Query after authentication' is an ordering relation and 'refresh credentials if the query fails' is recovery. Two separate text snippets might not express those links.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why do progressively merged event trees help locate evidence better than repeatedly rewriting a global summary under pressure?
MemForest: Efficient Agent Memory Management via EventTree Partitioning and Progressive Merging
Show answer and example
Answer: Event partitions preserve local time and causal structure. Merge only necessary nodes rather than disturbing all history. Queries propagate from semantic anchors through neighborhood links, including cross-event relations. Hierarchical summaries can still lose details; trees control loss and lookup paths without guaranteeing losslessness.
Illustrative example: Example: A failed payment and later refund are separate events linked across trees. Rewriting one global summary may blur their order.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
Where does the local token-Jaccard EventTree stop short of the paper's embeddings and LLM summarizer?
MemForest: Efficient Agent Memory Management via EventTree Partitioning and Progressive Merging
Show answer and example
Answer: Local CPU structures select trees by term Jaccard and run compression, checking partitioning and merging. They do not run author embeddings, an LLM summarizer, or long-horizon benchmarks. Structural correctness and fewer records do not establish semantic retrieval or QA capability.
Illustrative example: Example: Two nodes sharing meeting can match by Jaccard. Whether they describe the same event requires richer semantics not proved by this interface.
Sections in the Chinese source: 本地复现与边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
How does semantic-anchor neighborhood propagation differ from retrieving only the most similar node?
MemForest: Efficient Agent Memory Management via EventTree Partitioning and Progressive Merging
Show answer and example
Answer: A single match may be superficially related. Propagation can add context via temporal and cross-event links. This structure expands evidence but can also add noise, so control propagation range and budget. Neighboring nodes are not automatically correct evidence.
Illustrative example: Example: After matching a rescheduling proposal, read the neighboring confirmation to learn whether the new date took effect.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Why must long-term memory support both an overview and a return to specific events?
MemForest: Efficient Agent Memory Management via EventTree Partitioning and Progressive Merging
Show answer and example
Answer: Reading all raw history costs too much, but a single summary can omit the source of a decision. MemForest keeps events locally structured and progressively compresses under capacity pressure while preserving paths back to evidence. It balances scale with traceable detail.
Illustrative example: Example: Finding which meeting changed a plan last month requires both its event cluster and the actual confirmation message.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Core mechanism
Why start with action guidance and shift to observation enrichment later?
Environments as Scaffold: Enriching Feedback to Bootstrap Self-Evolving Agents in Long-Horizon Tasks
Show answer and example
Answer: Direct hints lower early exploration difficulty and yield learnable traces, but continued demonstration can make policy dependent. Later enriched observations retain evidence support while returning decisions to the agent, gradually reducing scaffolding. The phases serve different purposes and should not share constant strong hints.
Illustrative example: Example: Tell a beginner to find a key first. Later describe the lock and let the agent choose, rather than reveal the whole sequence.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Implementation details
Can the local first-step workflow be read from hidden AgentTask.plan?
Environments as Scaffold: Enriching Feedback to Bootstrap Self-Evolving Agents in Long-Horizon Tasks
Show answer and example
Answer: No. Use only a workflow explicitly supplied in public observation, then state summaries. Hidden plans never reach policy. No LLM training or real rollouts ran. The controlled path verifies isolation and schedules, not long-task learning from oracle-induced perfect scores.
Illustrative example: Example: A three-step reference plan in a fixture must not be disguised as an environment hint; that leaks the answer rather than enriches observation.
Sections in the Chinese source: 本地复现与边界. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Method comparison
Why evaluate scaffolded completion separately from autonomous completion after removing support?
Environments as Scaffold: Enriching Feedback to Bootstrap Self-Evolving Agents in Long-Horizon Tasks
Show answer and example
Answer: Scaffolding can simplify decisions or add evidence, so success reflects both policy and support. Removal tests whether training left transferable decisions. Report visible help and do not pool different information conditions. A phase-scheduling diagnostic does not replace real transfer after training.
Illustrative example: Example: If testing still tells the model to read config first, it measures following the hint. Autonomous evaluation requires discovering that action.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Basics
Why are fully unguided exploration and always giving answers both difficult for weaker long-horizon agents?
Environments as Scaffold: Enriching Feedback to Bootstrap Self-Evolving Agents in Long-Horizon Tasks
Show answer and example
Answer: Unguided exploration may yield too few successful experiences. Always providing correct actions can teach reliance on help. Feedback Scaffold changes support during training so early learning starts and later decisions are independent. Evaluate capability after removing the scaffold rather than count guided success as autonomy.
Illustrative example: Example: Demonstrate the next step at first, later give observations only. Continuing to choose actions during tests hides independent ability.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Core mechanism
Why does AutoLR not let the LLM directly save its preferred idea as the long-term champion?
AutoLR: Automating the Path from Research to Launch Review in Industrial Recommender Systems
Show answer and example
Answer: Noisy proposals and deliberations can compound when language judgments alone make KEEP decisions. The LLM does semantic reasoning; a deterministic controller handles budgets, metric extraction, guardrails, and state transitions. Only evidence passing the relevant experimental gates can update the champion, and a proxy win does not bypass later A/B testing or Launch Review.
Illustrative example: If an LLM suggests an optimizer whose offline benefit is unstable, log the proposal and failure and retain the old champion.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Implementation details
How does local AutoLR resume a disk ledger without using historical test scores for promotion?
AutoLR: Automating the Path from Research to Launch Review in Industrial Recommender Systems
Show answer and example
Answer: Proposals can see research evidence and the completed validation ledger. Run one real backward-pass check, train under a fixed budget, and evaluate test only once after selection. Restore candidate, review, and validation state, keep `online_authorized` false, and do not feed test results back as selection rewards.
Illustrative example: After an interruption, resume round three from prior validation records. If the final test regresses, report it instead of feeding the score into the proposal prompt and selecting again.
Sections in the Chinese source: 本地复现. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Method comparison
What trade-off does AutoLR's disagreement penalty make compared with choosing only for exploration value?
AutoLR: Automating the Path from Research to Launch Review in Industrial Recommender Systems
Show answer and example
Answer: Pure exploration favors unknown, risky ideas. Scoring existing evidence, exploration uncertainty, and expert disagreement gives conflict an explicit cost. Disagreement does not establish error; it calls for more evidence or lower budget priority. The local run did not beat the baseline in three rounds and could retain the champion.
Illustrative example: Of two equally novel ideas, test the broadly accepted one first under a tight budget and gather evidence about the contested one.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Basics
Why does AutoLR's long-term state machine distinguish offline evaluation, real A/B testing, and Launch Review?
AutoLR: Automating the Path from Research to Launch Review in Industrial Recommender Systems
Show answer and example
Answer: Offline results apply only under their data protocol; real user effects and deployment resource constraints need separate verification. The state machine separates proposals, experiments, reviews, and promotion, records failures and budgets, and increases the evidence at each stage. LLM reasoning or code generation cannot grant approval.
Illustrative example: An optimizer with better validation but excess memory stops offline. A candidate with real experiment evidence can enter launch review, not automatically become the live champion.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Core mechanism
Why can strongly co-occurring products be poor complements?
Beyond Co-purchase Relation: Evolution of Complementary Recommendations at Allegro
Show answer and example
Answer: Co-occurrence can reflect comparison or popularity. Complements jointly serve a need. Category evidence and adapters constrain compatibility without reading hidden target category.
Illustrative example: Comparing two cameras does not make them complements; a compatible lens may be.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Implementation details
Can Category Adapter inference read the hidden target category?
Beyond Co-purchase Relation: Evolution of Complementary Recommendations at Allegro
Show answer and example
Answer: No. Evaluate all candidate categories using their own features and train on allowed train pairs. Local data lacks true ComCat labels, so keep diagnostic status and negative outcomes.
Illustrative example: Telling the model the answer is a lens bypasses category selection.
Sections in the Chinese source: 本地复现. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Method comparison
How does complementary retrieval differ from similar-item retrieval?
Beyond Co-purchase Relation: Evolution of Complementary Recommendations at Allegro
Show answer and example
Answer: Similarity finds substitutes; complement retrieval finds functionally related items used together. Next-item hits are not true complement labels, so local Amazon negatives do not alone refute ComCat.
Illustrative example: Another tent is a substitute; a sleeping bag complements a tent.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Basics
Why build ComCat relations before category-constrained towers?
Beyond Co-purchase Relation: Evolution of Complementary Recommendations at Allegro
Show answer and example
Answer: Rules, people, LMs, and statistics define plausible complementary categories. Towers/adapters learn item-level matches within them. A category prior does not guarantee compatibility.
Illustrative example: Cameras and lenses complement each other, but mounts still need matching.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Core mechanism
Why doesn't separating KVMEM's address space from execution window mean all-history attention?
KVMem: Virtualizing Million-Token Agent Workspaces on a Consumer GPU
Show answer and example
Answer: Store paged KV across tiers, retrieve relevant blocks in attention space, and materialize a bounded native-window view. Missed evidence stays out. Local full backing cache and no complete async tiering mean no VRAM or speed claim.
Illustrative example: Load a few old KV pages per query, not every page at once.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Implementation details
Why not claim lower peak VRAM while retaining full backing cache?
KVMem: Virtualizing Million-Token Agent Workspaces on a Consumer GPU
Show answer and example
Answer: A bounded working set does not prove offload. Local Python retains backing cache and does not establish production kernels. Report working set, residency, movement, and end-to-end time separately.
Illustrative example: Reading 512 of 2048 resident slots still occupies all 2048.
Sections in the Chinese source: 本地复现, 复现边界. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Method comparison
How does paged KV reuse differ from retrieving text and prefilling it?
KVMem: Virtualizing Million-Token Agent Workspaces on a Consumer GPU
Show answer and example
Answer: Paged reuse materializes computed K/V; text retrieval usually computes K/V again. Position, storage, and consistency contracts differ, so retrieval count alone is not cost.
Illustrative example: Fetch cached KV versus fetch tokens and calculate their KV anew.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Basics
Why page long agent histories as KV?
KVMem: Virtualizing Million-Token Agent Workspaces on a Consumer GPU
Show answer and example
Answer: History can exceed memory and context; summaries may lose details. KVMEM stores computed KV in tiers, indexes relevant blocks, and materializes a native-window view.
Illustrative example: Many pages can exist while only several are loaded per execution.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Core mechanism
How does BeaconKV preserve plans that recent-only queries would evict?
Show answer and example
Answer: Maintain a small set of representative historical beacons, score KV across phases, and protect a recent window. Representatives approximate future access, not guarantee all distant evidence survives.
Illustrative example: Ten arithmetic steps may ignore an opening constraint that the final answer needs; a beacon can keep it important.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Implementation details
Why count the Python selector and backing cache in system costs?
Show answer and example
Answer: Selection and indexing may be expensive, and retaining full backing cache does not free GPU memory. Public checkpoint tests saw slowdown and quality loss; report quality, working set, and full cost separately.
Illustrative example: A 512-item attention set can still keep all KV and spend time scoring it.
Sections in the Chinese source: 本地复现, 复现边界. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Method comparison
How does beacon scoring differ from evicting by age?
Show answer and example
Answer: Age assumes newer is better. Beacon scoring uses query-space coverage and key matches, plus recent protection. It can still miss evidence and is not a future-query oracle.
Illustrative example: An old KV matching a planning query may outlast irrelevant old entries.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Basics
Why retain representative queries in addition to recent ones?
Show answer and example
Answer: Long reasoning may revisit an early plan. Beacons cover diverse attention directions for budgeted retention.
Illustrative example: The last few queries can miss the original instruction needed at the end.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why can shared team reward adapt both the reasoning and Meta-Skill policies?
Show answer and example
Answer: Skill content determines available procedures, and reasoning choices determine which skills get execution feedback. A joint objective connects both to task results; alternating updates reduces the risk of a revised library being unused or a revised policy being limited by old skills. Shared reward does not solve credit assignment for each edit.
Illustrative example: Example: If a long skill fails, Meta-Skill can shorten it while the reasoning agent learns when to invoke the revision instead of merely storing a new document.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
How should actual updates across three local seeds without promotion be interpreted?
Show answer and example
Answer: Both roles had nonzero gradients and parameter changes, but reset validation showed no positive gain. The skill candidate was not promoted and the baseline champion stayed. Old results that read evaluation labels are void, and legacy parsing is not capability. Local Qwen ToolRoute is not comparable with paper ALFWorld/WebShop scores.
Illustrative example: Example: Large parameter movement without validation gain proves execution and rejection work, not that CoSkill improves success.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
How does joint training differ from training retrieval over a fixed skill library?
Show answer and example
Answer: A fixed library changes only what is selected. Joint training also changes skill contents that may not suit the task, increasing nonstationarity and attribution difficulty. Role credit, isolated edits, and rejecting non-improving updates matter; library growth does not imply stronger teamwork.
Illustrative example: Example: Better retrieval cannot fix steps in the wrong order. Meta-Skill can fix the order, but must check other tasks for regressions.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Why is CoSkill's Meta-Skill Agent a learnable policy rather than a fixed script?
Show answer and example
Answer: It edits skill contents based on execution returns. The Reasoning Agent learns to select and use task and substep skills. Roles share a backbone and update alternately, adapting the library and action policy together. Verify that edits improve actual execution before keeping them.
Illustrative example: Example: When the reasoning role finds a missing recovery step, Meta-Skill proposes a change and validates after resetting the environment instead of permanently saving every reflection.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why does an atomic user profile help keep new preferences from overwriting old ones?
AtomRec: Evolving Atomic Memory for Agentic Recommendation
Show answer and example
Answer: Updates touch matching fact fields without rewriting the whole profile, reducing incidental deletions or semantic drift. Semantic collaborative edges and multi-hop paths connect recommendation evidence separately from preference values. Atomic fields still require conflict and time handling; old facts are not automatically permanent.
Illustrative example: Example: A new no-spicy-dinner preference changes that field only, preserving commute budget and breakfast preferences.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
What do local pre-target reconstruction and zeroing illegal links prevent?
AtomRec: Evolving Atomic Memory for Agentic Recommendation
Show answer and example
Answer: Rebuild memory from history strictly before each held-out target to prevent future behavior leakage. Generated links can cite only provided historical IDs; malformed or illegal output fails without reference-answer backfill. The small local test is negative, so executed data flow is not a recommendation or online gain.
Illustrative example: Example: Seeing the target product in later history makes the task easier. Remove it in the time-correct reconstruction, not just hide the prompt field name.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
How does a multi-hop evidence path differ from one collaborative user–item edge?
AtomRec: Evolving Atomic Memory for Agentic Recommendation
Show answer and example
Answer: A direct edge gives overall association. A multi-hop path can trace preference through a historical item attribute to a candidate. Semantic edges may still be wrong, so reachability is not causal proof of preference. Evaluate downstream ranking on a closed candidate set with target isolation.
Illustrative example: Example: Linking the lightweight attribute of previously liked shoes to a new pair is more traceable than saying they are often bought together, if the attribute link is grounded in actual history.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Why does a recommendation assistant need user memory that tracks changes and evidence?
AtomRec: Evolving Atomic Memory for Agentic Recommendation
Show answer and example
Answer: Behavior reflects long-term preferences, short-term intent, and item-specific feedback. A fixed profile is hard to update or explain. AtomRec evolves evidence at a fine granularity and uses history relations to explain candidates. A single click need not imply long-term preference, so retain context and provenance.
Illustrative example: Example: A gift perfume purchase should not make the system infer that the user personally prefers that scent long term.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Core mechanism
Why retain full context while supervising only a few largest positive gaps?
Extremely Sparse Supervision Incentivizes Reasoning Ability
Show answer and example
Answer: Teacher distributions depend on actual preceding reasoning. Sparse gradients don't remove conditioning. Limited-position gains are setup-specific, and no local full-parameter distillation was run.
Illustrative example: Score one key correction with prior calculations intact; deleting the prefix changes the task.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Implementation details
How distinguish unselected-token gradients from context?
Extremely Sparse Supervision Incentivizes Reasoning Ability
Show answer and example
Answer: Unselected positions have no direct loss but stay as conditioning; shared weights may still affect them. Local candidate tests aren't full Qwen/Llama training.
Illustrative example: Omitting step two's loss doesn't allow deleting it before scoring step three.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Method comparison
Why aren't sparse loss and shorter generation the same efficiency improvement?
Extremely Sparse Supervision Incentivizes Reasoning Ability
Show answer and example
Answer: Sparse loss reduces supervised positions while keeping full trajectories and often scores them to find key tokens. Shorter generation changes cost and state coverage.
Illustrative example: Generating a thousand tokens and training one still pays for all generation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Basics
Why study supervision at one or two positions in Sparse OPD?
Extremely Sparse Supervision Incentivizes Reasoning Ability
Show answer and example
Answer: Dense supervision may be redundant. Select important positive teacher–student differences to test concentration of useful information. A particular experiment doesn't prove one token suffices universally.
Illustrative example: Correcting one operator can matter more than imitating many connectives.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Core mechanism
Why protect full prompts but sample generated history independently per head?
Random Attention: Rethinking KV Cache Eviction for Efficient Reasoning
Show answer and example
Answer: Prompt preservation keeps task conditions and initial evidence fixed. Per-head random retention avoids scoring and gives diverse coverage, but may miss important reasoning and must fit the protected prompt budget. GPU reconstruction is not full task or kernel evaluation.
Illustrative example: Different heads can see different reasoning fragments while all still read the original question.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Implementation details
May the method drop prompt entries if B is smaller than prompt length?
Random Attention: Rethinking KV Cache Eviction for Efficient Reasoning
Show answer and example
Answer: Not under full prompt protection. B−|P| must be a valid generation budget. Raise the budget or report infeasibility rather than changing the method silently.
Illustrative example: A 100-entry prompt cannot fit untouched into 80 slots.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Method comparison
How does independent per-head sampling differ from a shared random set?
Random Attention: Rethinking KV Cache Eviction for Efficient Reasoning
Show answer and example
Answer: Head-specific sets vary evidence coverage; shared sampling applies one restriction everywhere. Average budgets can match while errors and coverage correlations differ.
Illustrative example: One head can keep position five and another eight.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Basics
Why omit KV importance scoring in Random Attention?
Random Attention: Rethinking KV Cache Eviction for Efficient Reasoning
Show answer and example
Answer: Scoring costs compute and can impose one shared ranking. Protect the prompt and sample generated positions uniformly per head as a simpler cache trade-off.
Illustrative example: Keep part of each head's history without scanning all tokens for importance first.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Core mechanism
Why use counterfactual neighbor addresses and a zero sink with hard Lngram v2 lookups?
Lngram v2: Latent N-Gram Memory with Interpretable Discrete Representations
Show answer and example
Answer: Binary addresses are discrete and ordinary indexing gives no smooth address gradient. Neighbor comparisons train address choice. GQA reads retrieved tokens alongside a zero sink so irrelevant memory need not be injected.
Illustrative example: A neighbor may be more useful than the addressed phrase; if neither helps, attend to zero.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Implementation details
Why include a zero sink instead of always using lookup results?
Lngram v2: Latent N-Gram Memory with Interpretable Discrete Representations
Show answer and example
Answer: It lets GQA reject irrelevant memory. Counterfactual adjacent addresses supply training signals; the integer index is not ordinarily differentiable.
Illustrative example: A query can attend to zero to keep an unhelpful lookup out of the residual.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Method comparison
How do latent addresses differ from text n-gram hashes?
Lngram v2: Latent N-Gram Memory with Interpretable Discrete Representations
Show answer and example
Answer: Backbone hidden projections carry context, while text hashes depend directly on discrete input fragments. They do not share an address space or interpretation.
Illustrative example: The same word in different contexts can yield different latent addresses.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Basics
Why turn hidden states into discrete n-gram addresses?
Lngram v2: Latent N-Gram Memory with Interpretable Discrete Representations
Show answer and example
Answer: Recent-state combinations index a memory table in expected O(1); multiple results feed back into the backbone. Explicit addressing has a trainable projection and is not lookup of literal strings only.
Illustrative example: Two binary hidden-state codes can form a 2-gram key.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Core mechanism
Why shouldn't a safety gate reject every action that leaves any residual violation?
SiLR: Structure-Preserving Admission and Process Reward for LLM Tool Agents
Show answer and example
Answer: Recovery can require intermediate states that remain partly unsafe. Rejecting all of them can deadlock the system. SiLR shadow-executes and requires every constrained branch not to worsen, allowing gradual improvement. A total can hide branch regressions, and shadow outcomes must match real execution contracts.
Illustrative example: Revoke one of two problematic permissions first. The system remains imperfect but improves; requiring all violations to disappear in one step can block recovery.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Implementation details
Can SiLR substitute an unreliable simulator for real effects during shadow execution?
SiLR: Structure-Preserving Admission and Process Reward for LLM Tool Agents
Show answer and example
Answer: Not as a safety claim. Admission requires deterministic, adequately covering verification. Shadow first, compare states, then execute. Without a real simulator, local checks establish only structured criteria and process signals.
Illustrative example: If shadowing omits deletion side effects, an unchanged vector can conceal actual risk.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Method comparison
How does product order differ from adding violation severities into one score?
SiLR: Structure-Preserving Admission and Process Reward for LLM Tool Agents
Show answer and example
Answer: Componentwise non-worsening prevents improvement on one branch from offsetting deterioration on another. A scalar can hide that exchange. Preserve constraints; a weighted sum is not an equivalent safety gate.
Illustrative example: Storage savings do not justify expanded permissions even if a combined score decreases.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Basics
Does SiLR shadow execution undo actions after real execution fails?
SiLR: Structure-Preserving Admission and Process Reward for LLM Tool Agents
Show answer and example
Answer: No. It evaluates proposed state effects on a shadow path before a gate admits real execution. Separate proposal assessment from real effects; executing first cannot establish prevention, especially for irreversible effects.
Illustrative example: Simulate which constraints a configuration edit touches before committing, rather than breaking the configuration and then attempting rollback.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
What confound is isolated by freezing task–harness trajectories and changing only advantage grouping?
What Does Multi-Harness RL Learn? Credit Assignment and Portability in Coding Agents
Show answer and example
Answer: Multi-harness training changes both data diversity and credit comparison groups. Comparing complete systems confounds them. Fixed trajectories let within-task comparison of cross-harness and within-harness normalization more directly test grouping. Report transfer differences and uncertainty on unseen minimal harnesses.
Illustrative example: Reuse trajectories for one repair task under two interfaces and recompute grouped advantages without resampling. This reduces attribution of sampling changes to the credit rule.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
Why does a held-out Cross−Within interval spanning zero fail to establish transfer?
What Does Multi-Harness RL Learn? Credit Assignment and Portability in Coding Agents
Show answer and example
Answer: A small positive point estimate is still compatible with no change; interpret the paper alongside its paired confidence interval. Old local gold-reading scores are invalid. Current checks cover grouping, gradients, and protocols, without Qwen RL, actual coding harnesses, or a sealed oracle. Counts are not SWE-bench capability.
Illustrative example: A 0.25 percentage-point gain with negative values inside the interval does not establish reliable superiority over Within. Do not omit the interval and report only the positive estimate.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
How do Within and Cross change the relative meaning of the same reward?
What Does Multi-Harness RL Learn? Credit Assignment and Portability in Coding Agents
Show answer and example
Answer: Within normalizes inside each task–harness group, comparing sibling trajectories in one environment. Cross normalizes across harnesses for the same task, including between-environment reward differences in advantages. It can learn environment preferences; portability needs paired tests on an unseen minimal harness, not just training means.
Illustrative example: A stronger-tool harness can earn generally positive Cross advantages because its rewards are higher. That does not imply better code repair on a minimal interface.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why separate base-model capability from execution-interface effects in multi-harness RL?
What Does Multi-Harness RL Learn? Credit Assignment and Portability in Coding Agents
Show answer and example
Answer: Different prompts, tools, and workflows change success on the same task; grouping changes the meaning of rewards. Multi-Harness RL examines their effects on learning and transfer. If the base, tasks, or samples also change, attribution is difficult. Control trajectories and evaluation definitions instead of calling all cross-harness gains capability growth.
Illustrative example: One harness fills arguments automatically while another requires the model to fill them. Success differences may reflect interface help rather than base reasoning.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Core mechanism
Why does CORAL project configurations onto a budget-feasible set and retain only recent experiment memory?
CORAL: An LLM-Native Harness for Production Recommender Systems
Show answer and example
Answer: LLM proposals can contain negative allocations or exceed quotas. Projection creates executable candidates before real feedback informs the next round. Short memory helps adapt to change and limits stale guidance but does not justify deleting audit records. Feasible proposals still need quality validation.
Illustrative example: If three retrieval allocations exceed a request cap, adjust them before testing. After traffic shifts, an old optimum should not automatically dominate.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Implementation details
Why not clip each CORAL allocation separately when projecting onto nonnegative values under a total budget?
CORAL: An LLM-Native Harness for Production Recommender Systems
Show answer and example
Answer: Per-coordinate clipping fixes negative values but not the coupled sum bound B. Joint projection enforces both. Store the executed configuration with its observations, not the infeasible proposal. The five-round local run verifies constraints and loop executability only.
Illustrative example: Three individually legal allocations of 6 total 18, exceeding a cap of 10. Adjust together and record what actually ran.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Method comparison
When is CORAL's short memory preferable to retaining the all-time best configuration?
CORAL: An LLM-Native Harness for Production Recommender Systems
Show answer and example
Answer: Recent feedback is more useful when the environment shifts; all-time maxima assume greater stability. CORAL updates from recent observations under projected constraints, potentially forgetting valuable long-run experience. A small proxy score cannot resolve the trade-off universally.
Illustrative example: A holiday ad mix can be poor afterward. Recent evidence should matter, while durable safety constraints should not be forgotten.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Basics
Why does CORAL require structured objectives and budgets rather than free-form LLM optimization advice?
CORAL: An LLM-Native Harness for Production Recommender Systems
Show answer and example
Answer: A system must verify, execute, and compare configurations. Budgets define legal allocations and objectives translate observations into feedback. A successful loop is not by itself production causal evidence.
Illustrative example: Asking for more retrieval compute is not executable until channels and amounts are specified and checked against the cap.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why put scalable ranking, hierarchical SID generation, and offline reason tokens in one framework?
Show answer and example
Answer: Ranking discriminates finely, SID generation spans items and pages, and offline tokens carry prepared reasoning online. Inspect generation paths, list objectives, and reasoning interfaces separately.
Illustrative example: Precompute interest explanations, generate a page from them, and rank without live long reasoning.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Does local path confidence reproduce production CCFormer?
Show answer and example
Answer: No. The public core omits private SID catalog, checkpoints, and live generation. Align level probabilities and list scores explicitly; proxy fusion is not deployment evidence.
Illustrative example: A working multi-level code score is not a complete page generator or ranker.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
What is the trade-off between offline reason tokens and live inference?
Show answer and example
Answer: Offline reduces critical-path latency but can age and miss abrupt context changes. Live reasoning is flexible but expensive. Compare inputs and update cadence.
Illustrative example: Yesterday's reasoning can miss today's trending topic.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Why combine ranking, generation, and offline reason in TGR?
Show answer and example
Answer: Industry needs scalable scoring and item/list generation at controlled latency. Offline reasoning supplies inputs without costly online rollouts; distinct modules are not one attention equation.
Illustrative example: Prepare reasoning offline, then generate and rank live candidates.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why separate heavy user encoding and light candidate interaction in ReST compute-once, decode-many?
Show answer and example
Answer: Long-history gated attention, order, and time intervals are shared user compute. Candidate cross-decoders read relevant evidence without rerunning history. Dual gates suppress noise, but only candidate-independent state is reusable and history/version changes invalidate it.
Illustrative example: Illustration: Encode history once for 200 films, then let each query it; never share one film's interaction result as general user state.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
What dependencies matter before reusing ReST encoder caches across requests?
Show answer and example
Answer: Same-request hU reuse requires unchanged history and version. New events, relative time encodings, or model updates can invalidate cross-request state. Candidate decoder outputs remain private to their candidates. Local structure does not prove production QPS or cache lifecycle.
Illustrative example: Illustration: After a user buys a camera, update history instead of reusing the old summary forever because user ID matches.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
What evidence does ReST gating preserve compared with truncation?
Show answer and example
Answer: Truncation discards events; gating can suppress noise across broader history while retaining distant evidence for candidate decoding. It can suppress important items too and still incurs input cost. A small local regression does not show gating beats recent-window baselines.
Illustrative example: Illustration: Keep a small weight on old photography and downweight accidental news; truncation could remove the useful photography event entirely.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why combine RoPE and relative time in ReST?
Show answer and example
Answer: Position and elapsed time differ: adjacent events can be a minute or a year apart. RoPE represents order, time encoding adds cadence, and dual gates filter noise. Not all periodicity is captured and large gaps are not always irrelevant.
Illustrative example: Illustration: Camera views five minutes apart suggest one comparison; views six months apart may mark a new need.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Core mechanism
Why backpropagate through a negative caption's current likelihood if a frozen model generated it?
From Visual Cues to Spoken Narration: Rethinking Audio Description
Show answer and example
Answer: Freezing its generator fixes the negative text, not the trainable model's scores. DRL compares mean target-token log likelihoods for positive and negative texts with the same visual input. Gradients improve their relative preference; detaching the negative score changes the contrastive optimization.
Illustrative example: For the same scene, one caption says a person walks and the other says they run. The strings are fixed, but the trainable model still changes their relative scores.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Implementation details
Can SFT and DRL use one shared length normalization?
From Visual Cues to Spoken Narration: Rethinking Audio Description
Show answer and example
Answer: No. The pinned SFT sums valid target tokens per sample before the batch mean. DRL averages token log probabilities separately in each description. Both exclude prompt and padding; freezing the negative generator does not stop gradients through the current model's negative-caption scores.
Illustrative example: For 10 positive tokens and 20 negative tokens, DRL compares their means. Raw sums would additionally penalize the longer description.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Method comparison
What does DRL add beyond SFT on positive descriptions?
From Visual Cues to Spoken Narration: Rethinking Audio Description
Show answer and example
Answer: SFT raises correct target-token likelihood; DRL explicitly separates positive and negative mean log probabilities. This adds a relative signal whose meaning depends on how negatives are built. Trivial negatives do not automatically teach detailed visual grounding.
Illustrative example: Raising the likelihood of “She sits down” need not lower the model's score for the wrong “She runs away.” The relative loss constrains the pair directly.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Basics
Why pair the negative caption with the same visual cue and prompt?
From Visual Cues to Spoken Narration: Rethinking Audio Description
Show answer and example
Answer: Matching conditions isolates whether the descriptions suit the intended audio-description task. Changing image or prompt could instead change task difficulty. Offline negative captions are training material, not an extra deployment generation step.
Illustrative example: Score “She sets down the cup” and an unrelated-action caption with the same film clip and instruction.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Core mechanism
Why use one-to-one matching in training and presence/NMS in serving?
SetMIR: Multi-Interest Retrieval as Set Prediction
Show answer and example
Answer: Matching prevents training collapse onto one positive. Presence activates relevant slots per request; NMS removes redundant retrieval calls. These solve different problems.
Illustrative example: Activate photography and camping, then suppress two nearly identical camera queries.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Implementation details
Which SetMIR queries actually dispatch ANN?
SetMIR: Multi-Interest Retrieval as Set Prediction
Show answer and example
Answer: Only those passing presence and surviving query NMS. The public CPU path does not reproduce private histories, production Transformer, or ANN dispatch.
Illustrative example: Ten available slots need not make ten calls.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Method comparison
How does one-to-one set matching differ from independent slot choices?
SetMIR: Multi-Interest Retrieval as Set Prediction
Show answer and example
Answer: Joint assignment limits slots from repeatedly explaining one easy positive. It needs reliable target sets; unique assignments do not prove complete interest discovery.
Illustrative example: Assign slots to sports and photography rather than both greedily choosing sports.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Basics
Is a SetMIR query a user-entered search string?
SetMIR: Multi-Interest Retrieval as Set Prediction
Show answer and example
Answer: No. It is a generated interest vector or slot for ANN. Multiple slots capture possible interests; serving rules decide which activate.
Illustrative example: Opening recommendations can produce sports and photography vectors without a typed query.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why add co-engagement contrastive learning rather than training CAMIE item vectors only on image-text similarity?
CAMIE: Co-Engagement-Aware Multimodal Item Embeddings for Snap Dynamic Product Ads Retrieval
Show answer and example
Answer: Content resemblance does not guarantee a consumption relationship. CAMIE uses item pairs jointly engaged with in a user journey for symmetric contrastive supervision, shaping multimodal embeddings for recommendation. The shared encoder supports variable modality availability. Co-engagement still reflects exposure and context, not universal complementarity or causal demand.
Illustrative example: Similar furniture can serve incompatible uses, while visually different desks and chairs are often considered together. Behavior supervision adds that relation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
Why compute both directions of CAMIE's symmetric in-batch InfoNCE?
CAMIE: Co-Engagement-Aware Multimodal Item Embeddings for Snap Dynamic Product Ads Retrieval
Show answer and example
Answer: One direction uses a as query and its paired b as positive among the batch's b values. Swap sides to supervise both encoders and average the losses. Pairs must share the intended journey relation, temperature controls sharpness, and duplicated positives should not blindly count as negatives.
Illustrative example: Learn camera-to-lens and lens-to-camera matching. A duplicate row for the same positive lens would otherwise give contradictory supervision.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
How do CAMIE's co-engagement positives differ from generic image-text contrastive positives?
CAMIE: Co-Engagement-Aware Multimodal Item Embeddings for Snap Dynamic Product Ads Retrieval
Show answer and example
Answer: Image-text positives typically pair modalities of the same content. Co-engagement pairs different items that share user interactions in a journey, encoding potential complementarity or sequential needs. Exposure still affects which items co-occur, so it is not an unbiased complementarity label.
Illustrative example: A tent and sleeping bag may convert together despite very different images; two popular items may merely co-occur because both receive heavy exposure.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why should one CAMIE checkpoint support image-text and text-only products?
CAMIE: Co-Engagement-Aware Multimodal Item Embeddings for Snap Dynamic Product Ads Retrieval
Show answer and example
Answer: Real catalog modality availability varies. A shared interface lets products without pictures enter the same retrieval space instead of requiring separate models. Joint training and behavior alignment support matching across inputs, but a missing image removes information, so accuracy need not be equal.
Illustrative example: A title-only launch and an item with images and attributes can both be indexed, but the former has no visual details to recover.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
After generating a pseudo skill, why does CSI recheck capability fit before execution?
Imagine Before Acting: Prospective Skill Identification for LLM Agents
Show answer and example
Answer: The pseudo skill turns a goal into a procedural retrieval proxy. It may locate related documents, but it is not verified and may not fit the subgoal. CSI rechecks evidence, constraints, and the next bounded action, linking retrieval to correct use without assuming imagined steps are executable.
Illustrative example: Example: A pseudo skill can find a web download procedure. Before executing, check whether the current page requires login instead of following a similar document title blindly.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
Does CSI check every five reasoning turns or every five executed actions?
Imagine Before Acting: Prospective Skill Identification for LLM Agents
Show answer and example
Answer: It counts actually executed actions. Before the next action, recheck the current subgoal, evidence, and constraints; MATCH or PARTIAL_MATCH loads the real procedure first. Loading is not tool success, and invalid pseudo skills are not heuristically fabricated. Report loads, executions, and final passes separately.
Illustrative example: Example: Three reflections without tool execution do not advance the action counter or trigger the periodic check early.
Sections in the Chinese source: 核心公式, 本地复现, 数据协议、结果与边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
What is HSR capability round-robin trading off against global top-score selection?
Imagine Before Acting: Prospective Skill Identification for LLM Agents
Show answer and example
Answer: Global top scores can all come from one easily matched capability, missing other requirements. HSR ranks within capability and cycles through rows with deduplication to preserve coverage. Min–max normalization within candidate sets makes scores set-dependent, not calibrated success probabilities across tasks.
Illustrative example: Example: A task needs cleaning, plotting, and export. Round-robin gives each a candidate instead of using all five slots for cleaning.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Why is a pseudo skill a retrieval proxy rather than a verified procedure?
Imagine Before Acting: Prospective Skill Identification for LLM Agents
Show answer and example
Answer: The model imagines capabilities and procedure-like descriptions to bridge task wording and skill documents. These descriptions have not been executed and may omit steps or violate conditions. Load and validate the real procedure before acting; a complete-looking description is not proof.
Illustrative example: Example: A pseudo skill says export and verify files in bulk. It helps find the export skill, but real parameters, permissions, and checks come from the actual documentation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Core mechanism
Why does Friend-GNN need both multi-hash user representations and temporal-prefix neighbor sampling?
Show answer and example
Answer: Multi-hash tables reduce storage but do not prevent time leakage. Temporal CSR sorts edges so queries sample only pre-cutoff neighbors before graph aggregation. Verify memory and observational legality separately from ranking scores.
Illustrative example: Tuesday's friend recommendation cannot use Friday's new connection, no matter how compact the user embeddings are.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Implementation details
Why sort per-user neighbor times before taking a historical prefix in temporal CSR?
Show answer and example
Answer: Binary search needs sorted timestamps to locate cutoff `τ−Δ`, then sample only its legal prefix. Use `lower_bound` semantics to determine cutoff inclusion rather than claiming the current full graph was filtered.
Illustrative example: Unsorted January, February, and March edges can cause a binary search for a February cutoff to admit March.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Method comparison
Why do three independent concatenated hashes mitigate rather than eliminate collisions?
Show answer and example
Answer: A single hash gives same-bucket users one embedding. Multiple addresses reduce the chance all buckets match, and projection combines them. Partial sharing and complete collisions remain; this is not an independent ID table.
Illustrative example: Two users with the same first bucket can differ in the other two, but identical triples remain indistinguishable by this encoding.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Basics
Why address both user-ID storage and temporal edge visibility in Friend-GNN?
Show answer and example
Answer: Massive graphs make per-user tables expensive; shared hashes compress identities. Edges after prediction expose outcomes, so constrain time separately before GNN propagation. Neither property establishes recommendation accuracy.
Illustrative example: A billion users can share hash tables, but predicting yesterday still cannot sample today's friend links.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Core mechanism
Why mix weak prefixes even though they can be wrong?
Boosting LLM Exploration via Weak-Model Guidance in RLVR
Show answer and example
Answer: They provide uncommon exploration starts. Entropy truncation limits unstable continuations and native traces maintain coverage. Hints aren't labels; use independent verifier. Frozen proxy isn't dual-model training.
Illustrative example: An alternate decomposition can be tried but still receives failure if validation fails.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Implementation details
Is the local frozen-reference proxy actual dual-checkpoint RLVR?
Boosting LLM Exploration via Weak-Model Guidance in RLVR
Show answer and example
Answer: No. Candidate distributions simulate guidance. Record mixture proportions and prefix boundaries; keep hints out of independent test. No real two-model generation.
Illustrative example: Numeric alternative starts don't measure linguistic weak-prefix quality.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Method comparison
How does prefix guidance differ from SFT on complete weak answers?
Boosting LLM Exploration via Weak-Model Guidance in RLVR
Show answer and example
Answer: It supplies only a starting point; target generates and learns from reward. Native/guided mixtures and entropy truncation control coverage instead of trusting weak solutions wholesale.
Illustrative example: Give an opening idea without asking the student to copy a wrong conclusion.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Basics
Why can weak-model prefixes improve strong exploration?
Boosting LLM Exploration via Weak-Model Guidance in RLVR
Show answer and example
Answer: They move the target to states its own policy rarely visits, expanding reasoning coverage. Prefixes can be wrong; independent verification still defines reward.
Illustrative example: Try continuing an alternative decomposition and check the result.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Core mechanism
Why keep evidence frames teacher-only and full video for student?
Show answer and example
Answer: Human-labeled frames give focused supervision, weighted by token dependence. Student must handle distractions as deployed; supplying frame labels changes the task. No-video diagnostics aren't vision evidence.
Illustrative example: To answer who picked up the cup first, student still needs to locate the relevant segment.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Implementation details
Does local Video-OPSD smoke include video frames?
Show answer and example
Answer: No. Independent reward axes proxy evidence. Available checkpoints don't imply trained video model. Align teacher targets to student prefixes and keep annotations out of deployment.
Illustrative example: A synthetic process score doesn't show the model watched a video.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Method comparison
How do evidence-dependent weights differ from equal token KL?
Show answer and example
Answer: Emphasize positions influenced by visual evidence, not all style tokens equally. Dependency isn't factual certainty. No-video smoke doesn't validate paper training time.
Illustrative example: An action word can depend more on frames than a connective.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Basics
Why does Video-OPSD teacher see evidence frames while student sees full video?
Show answer and example
Answer: Privileged frames focus teacher supervision, while student learns to find relevant evidence amid distractions and does not receive annotation at inference.
Illustrative example: Teacher can see the cup pickup moment; student must find it within the full activity.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Core mechanism
What do temperature expansion, trajectory selection and token gating each solve?
Preserving General Capabilities during Domain Specialization with Uncertainty-Calibrated MOPD
Show answer and example
Answer: Expand candidate coverage, favor trajectories with dense positive advantages, then gate individual updates with entropy-calibrated probabilities. More samples don't guarantee correct supervision; no full teachers locally.
Illustrative example: Sample varied solutions, select one with useful steps and reinforce only reliable tokens.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Implementation details
What does stochastic UC-MOPD gating need to log, and were checkpoints loaded?
Preserving General Capabilities during Domain Specialization with Uncertainty-Calibrated MOPD
Show answer and example
Answer: Log probabilities, seed and valid positions so random draws aren't deterministic decisions. No general/domain teachers loaded; formula checks don't prove retained general capabilities.
Illustrative example: Track each differing random mask, not just report the best draw.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Method comparison
How do positive-advantage density and CLL gating work together?
Preserving General Capabilities during Domain Specialization with Uncertainty-Calibrated MOPD
Show answer and example
Answer: Density selects useful trajectories; CLL controls local tokens inside selected traces. Both are required and confidence isn't truth.
Illustrative example: A generally useful solution can still contain positions not worth reinforcing.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Basics
Why expand temperature coverage before UC-MOPD selection?
Preserving General Capabilities during Domain Specialization with Uncertainty-Calibrated MOPD
Show answer and example
Answer: Ordinary sampling may yield few positive-credit positions. Dual temperatures broaden exploration, then trajectory density/token reliability filter it. More candidates aren't all correct.
Illustrative example: Try another solution first, then check whether its steps help.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
Why use different objectives for majority agreement and disagreement?
TTPO: Test-Time Policy Optimization
Show answer and example
Answer: Copying consensus can spread errors. Self-distill agreeing replies, apply group RL to dissent, and filter already converged positions and high-confidence errors. No gold or claim that votes are truth.
Illustrative example: Don't delete the dissenting answer outright; handle it under the relevant objective and reliability filters.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
Does local TTPO perform real test-time sampling and Qwen updates?
TTPO: Test-Time Policy Optimization
Show answer and example
Answer: No. It tests branch objectives and token selection only. Fixed candidates are not real unlabeled adaptation. Keep gold hidden before branch assignment.
Illustrative example: Supplying answers for formula checks does not mean the model generated them on benchmark prompts.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
What safeguards does TTPO add instead of treating the majority answer as gold?
TTPO: Test-Time Policy Optimization
Show answer and example
Answer: Asymmetric objectives filter converged tokens and confident errors. It recognizes that consensus is not truth; filter quality matters and consensus isn't independent validation.
Illustrative example: A group can confidently agree on the same wrong answer.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
Why train TTPO majority-agreeing and dissenting trajectories separately?
TTPO: Test-Time Policy Optimization
Show answer and example
Answer: The majority may be wrong, and dissent provides a different signal. Use self-distillation for agreement and grouped RL for disagreement rather than one unlabeled pseudo-target.
Illustrative example: A majority answer and an alternative path do not share the same imitation objective.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Core mechanism
Why align milestones with LCS-F1 instead of checking the same symbols?
Show answer and example
Answer: Identical sets can break reasoning order. LCS preserves relative order and precision/recall detect redundancy and omissions, providing dense feedback. Extraction can err; local candidate axes are not a natural-language extractor.
Illustrative example: Substituting conditions before solving differs from writing the result then inventing conditions.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Implementation details
Does local SPEAR extract natural-language milestones?
Show answer and example
Answer: No. Candidate process/format axes proxy milestones for LCS-F1. Teacher-text extraction is separate; known symbolic labels don't prove extraction.
Illustrative example: Supplying symbols directly cannot test extracting them from a long proof.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Method comparison
How does LCS-F1 differ from unordered overlap?
Show answer and example
Answer: LCS respects order; F1 measures coverage and redundancy. Reordered dependencies can retain the same set yet deserve different credit.
Illustrative example: Putting a conclusion before its premises can look identical to set overlap.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Basics
Why convert teacher reasoning to symbolic milestones in SPEAR?
Show answer and example
Answer: Terminal reward is sparse and neural process judges costly. Domain milestones permit ordered alignment and dense progress, if extraction preserves important reasoning.
Illustrative example: Match key equations rather than copying all teacher wording.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
Why are Merge, Mix RL and MOPD different fusion operations?
Consolidating RLVR Capabilities Across Domains: A Deep Dive into Fusion Paradigms
Show answer and example
Answer: Merge combines expert parameter increments, Mix RL trains on combined data, and MOPD uses expert guidance with data. Availability and budget determine feasible routes. Similar means or a probability-vector proxy do not establish lossless checkpoint fusion.
Illustrative example: Experts alone permit discussing parameter merging; data without experts cannot run the same distillation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
Why define Merge task vectors relative to a common base?
Consolidating RLVR Capabilities Across Domains: A Deep Dive into Fusion Paradigms
Show answer and example
Answer: Expert minus base must share parameter coordinates and reference. Different bases/layouts don't express comparable deltas. Local candidate merging isn't checkpoint merging.
Illustrative example: Experts with different vocabularies or tensor layouts cannot simply average deltas.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
Why doesn't similar average performance imply per-domain equivalence?
Consolidating RLVR Capabilities Across Domains: A Deep Dive into Fusion Paradigms
Show answer and example
Answer: Averages can cancel larger task gaps. Compare target domains, available experts/data and compute, not only mean scores or similar formulas.
Illustrative example: Better math and worse code can yield the same mean without meeting the same needs.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
What resources do Merge, Mix RL and MOPD reuse in RLVR Fusion?
Consolidating RLVR Capabilities Across Domains: A Deep Dive into Fusion Paradigms
Show answer and example
Answer: Merge reuses expert weight deltas; Mix RL reuses training data; MOPD distills from experts using data. Resource requirements differ, so averages cannot choose a route alone.
Illustrative example: Without original training data, available routes differ even if expert weights are present.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Core mechanism
Why can dividing by beta change gradients without moving Normalized DPO's optimum?
Disentangling Optimization Scale from Preference Scale in DPO
Show answer and example
Answer: At fixed positive beta, subtracting a constant and multiplying by a positive scalar preserve optima. Ordinary DPO couples preference-curve scale with update size; dividing centered softplus by beta removes an extra beta in the margin gradient.
Illustrative example: Reducing beta can weaken ordinary updates; normalization separates curve scale from step-size effects.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Implementation details
How can normalized DPO avoid zero-over-zero for tiny beta?
Disentangling Optimization Scale from Preference Scale in DPO
Show answer and example
Answer: Compute centered softplus minus log 2 stably or use its small-beta limit. The gradient is negative sigmoid and does not vanish from an extra beta factor. Test the limit rather than substituting zero in the quotient. Candidate checks are not full training.
Illustrative example: Confirm finite limiting loss and gradient instead of catching exceptions and returning zero.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Method comparison
Is Normalized DPO equivalent to changing the learning rate?
Disentangling Optimization Scale from Preference Scale in DPO
Show answer and example
Answer: Centering and dividing by beta rescales loss gradients. Learning rate scales the optimizer's update. They can be related under simple conditions, but momentum/adaptive statistics can change the trajectory.
Illustrative example: A second-moment optimizer need not maintain identical state under external step-size and loss scaling.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Basics
Is Normalized DPO's margin a human score gap or a model-relative preference?
Disentangling Optimization Scale from Preference Scale in DPO
Show answer and example
Answer: It is the model-side value in the loss. Data identify the winner; policy and reference probabilities determine the margin and beta sets its scale in the preference curve. Normalization does not rewrite labels.
Illustrative example: A human says A beats B without saying by three points; current probabilities still give a margin.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Core mechanism
Why swap redundant donors for orphans rather than raise TwinKV budget?
TwinKV: A Composable Repair Pass for KV Cache Eviction via Pairwise Key Redundancy
Show answer and example
Answer: The base eviction can keep duplicates and lose unique evidence. Recover dropped keys without similar retained substitutes by removing redundant retained keys with distant twins. Protect sink/recent items. Key similarity is not semantic equivalence, and reconstruction is not generation.
Illustrative example: Replace one of two repeated explanations with a unique date at the same cache size.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Implementation details
Why exclude local twins and sink/recent protected entries from donors?
TwinKV: A Composable Repair Pass for KV Cache Eviction via Pairwise Key Redundancy
Show answer and example
Answer: Find similar retained twins outside the local window and never sacrifice protected entries. Swap only min(orphans, donors) and preserve |R′|=|R|.
Illustrative example: Nearby similar keys do not automatically qualify as distant redundancy.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Method comparison
How does TwinKV compose with StreamingLLM or H2O?
TwinKV: A Composable Repair Pass for KV Cache Eviction via Pairwise Key Redundancy
Show answer and example
Answer: It repairs a wrapped policy's result rather than replacing its score. Compare the same initial policy and KV budget with repair on or off.
Illustrative example: StreamingLLM and StreamingLLM+TwinKV retain equal counts, isolating swaps.
Sections in the Chinese source: 背景与主要改动, 本地复现. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Basics
Why repair retention under a fixed budget?
TwinKV: A Composable Repair Pass for KV Cache Eviction via Pairwise Key Redundancy
Show answer and example
Answer: A base policy can keep redundant keys and drop unique ones. Donor–orphan swaps address gaps without adding entries.
Illustrative example: Trade one near duplicate for unique older evidence.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Core mechanism
Why separate image scaling and question-dependent token extraction in PACE?
PACE: A Unified Condense-and-Extract Paradigm for Fast VLM Inference
Show answer and example
Answer: Scaling controls input encoding cost; extraction controls what continues downstream. Late token dropping cannot reclaim prior encoding. A shallow preview estimates difficulty, and extraction mixes language and visual attention by confidence rather than using one fixed rule.
Illustrative example: Shrink a blank image more than a dense menu, then use the question to select relevant regions.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Implementation details
Does the real-checkpoint run fully validate DDAE?
PACE: A Unified Condense-and-Extract Paradigm for Fast VLM Inference
Show answer and example
Answer: No. The small real-checkpoint smoke mainly exercises APC. DDAE is checked with equations and unit tests, without the original model fork or kernels. APC excludes diagonal self-similarity and picks the top-K deviations from the local mean. Correct compression direction does not establish lossless quality.
Illustrative example: Five correct out of eight before and after compression is a small-sample observation, not verification of DDAE in generation.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Method comparison
How does DDAE differ from an equal-weight attention average?
PACE: A Unified Condense-and-Extract Paradigm for Fast VLM Inference
Show answer and example
Answer: It converts the dispersions of two normalized attention maps into confidence weights using temperature softmax. Weights can favor different streams depending on evidence. Dispersion is a method signal, not a calibrated probability of correctness.
Illustrative example: Concentrated language attention and diffuse vision attention may favor language; another question can reverse the weights.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual evidence budgets: tokens, frames, and on-demand revisiting
Basics
Why can't PACE infer equal vision budgets from equal image sizes?
PACE: A Unified Condense-and-Extract Paradigm for Fast VLM Inference
Show answer and example
Answer: Equal-sized images can differ in global semantic density and local detail. APC estimates both off-diagonal feature similarities and top-K details that differ from background. Pixel count is not a substitute, and budget signals do not guarantee correctness.
Illustrative example: A blank wall and a densely printed page at the same resolution need not preserve the same amount of detail.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Core mechanism
Why doesn't CritICL need the strong model to fail first at deployment?
CritICL: Inference-Time Weak-to-Strong Generalization from Small Language Model Failure Modes
Show answer and example
Answer: Offline weak-model failures become reusable critiques. Static mode uses common patterns and dynamic mode retrieves question-relevant ones before one strong-model generation. Transfer patterns, not weak answers; relevance and coverage matter.
Illustrative example: Retrieve a unit-conversion reminder before answering, without reading test gold to craft it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Implementation details
Why not select critiques with the current question's correct answer?
CritICL: Inference-Time Weak-to-Strong Generalization from Small Language Model Failure Modes
Show answer and example
Answer: That leaks evaluation targets into prompts. Dynamic retrieval uses only input similarity to the existing bank and then generates once. Offline failures and current hidden gold stay separate.
Illustrative example: Looking up the solution before picking the critique that fixes it is not deployable retrieval.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Method comparison
How do static and dynamic critique selection differ?
CritICL: Inference-Time Weak-to-Strong Generalization from Small Language Model Failure Modes
Show answer and example
Answer: Static uses frequent global failures; dynamic retrieves relevant CritBank entries for the current input. Both act before generation, but only dynamic adds input-dependent retrieval.
Illustrative example: A math question can retrieve calculation reminders different from those for other requests.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Basics
Why build CritBank from weak-model errors?
CritICL: Inference-Time Weak-to-Strong Generalization from Small Language Model Failure Modes
Show answer and example
Answer: Cluster recurring failures offline into reusable reminders for stronger models. This moves reflection cost upstream and avoids requiring a first failure on each deployed request.
Illustrative example: Add a common missed-unit-conversion reminder before a new answer.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Core mechanism
Why retain complete context but apply SFT loss only to selected semantic segments?
SWE-Prime: Fewer Trajectories, Better Performance
Show answer and example
Answer: Unselected spans may be necessary causal context for later behavior even if redundant, dangerous, or unhelpful to imitate. Masking separates readable context from training targets. The model sees the real process while learning useful, learnable actions; not every token in a successful trajectory deserves supervision.
Illustrative example: Example: Keep a failed dangerous command to explain a repair, without rewarding the model for generating that command.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Implementation details
Do local filtering results show a code LLM was trained or SWE-bench ran?
SWE-Prime: Fewer Trajectories, Better Performance
Show answer and example
Answer: No. Local filtering strategies share an observation and tool interface without training a code model or running official SWE-bench. Verify that loss masks cover only selected spans, context stays complete, and data splits remain isolated. Small tool-routing metrics are not code-repair gains.
Illustrative example: Example: More stable local routes after removing risky segments do not reproduce the paper's outcome with one tenth of its code traces.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Method comparison
What different data-quality problems do whole-trajectory and within-trajectory segment selection solve?
SWE-Prime: Fewer Trajectories, Better Performance
Show answer and example
Answer: Trajectory selection chooses useful complete experiences by outcome, process, and representativeness. Segment selection chooses contributing, learnable, lower-risk behavior within them. Good traces can contain redundancy, and good fragments can come from failed overall runs. Sample selection and supervision are distinct.
Illustrative example: Example: Keep a passing repair's complete execution as context, but exclude irrelevant dependency installation from imitation targets.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Basics
Why might a successful trajectory still be a poor full SFT target?
SWE-Prime: Fewer Trajectories, Better Performance
Show answer and example
Answer: Success can include redundant exploration, risky commands, or steps the current model cannot learn. SWE-Prime filters trajectories by process, outcome, and representativeness, then selects segments by contribution, learnability, and risk. Outcome success does not endorse every token.
Illustrative example: Example: A bug fix ultimately passes after accidentally deleting and restoring a file. Do not teach deletion as a good action.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why is inserting referenced files near first use more complete than pretraining on skill instructions alone?
SPT: Skills as Pre-Training Data for Agentic Language Models
Show answer and example
Answer: Skill packages put argument rules or examples in companion files. Inlining at first reference exposes real dependencies at the right position in causal context rather than training on dangling references. Mixing with general text limits distribution shift. Reading skill documents is not validated execution.
Illustrative example: Example: 'Use schema.json for arguments' does not teach the field constraints if the schema is absent; inserting it near the reference supplies them.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
Why retain general text and control its mixture with skill data in mid-training?
SPT: Skills as Pre-Training Data for Agentic Language Models
Show answer and example
Answer: Training data changes both what a model learns and the text distribution it sees. An explicit mixture coefficient α controls the share of procedural packages. Replacing all training with skill documents does not leave other capabilities unchanged by assumption. Local assembly and mixture tests cannot establish general capability retention or execution skill.
Illustrative example: Example: A batch includes instructions and general prose. Raising the skill ratio changes the training emphasis, which requires training evaluation, not a file-count check.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
How does skill mid-training differ from retrieving skill documents only at deployment?
SPT: Skills as Pre-Training Data for Agentic Language Models
Show answer and example
Answer: Mid-training writes procedural patterns into weights, so the same document need not be retrieved each time. Runtime retrieval puts up-to-date skills explicitly in context but costs tokens. They can complement each other. Training on skill text does not prove tool execution; verify a real policy and environment.
Illustrative example: Example: A model may remember to verify before submitting, while a changed API argument still needs current runtime documentation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Why can multi-file skill packages serve as mid-training data for a causal LM?
SPT: Skills as Pre-Training Data for Agentic Language Models
Show answer and example
Answer: They contain reusable procedures and referenced materials. Predicting this text before post-training can build workflow priors. This is not a single execution trace or real tool use during training; downstream SFT or RL still must turn document patterns into behavior.
Illustrative example: Example: Instructions with a configuration example and helper script teach a reusable method, not the accidental full tool log of one task.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Core mechanism
Why must an experience skill beat the incumbent on a holdout rather than just come from a successful audit trace?
RedEvoAgent: Automatic Red-Teaming Agent with Experience-Driven Skill Evolution
Show answer and example
Answer: The original success supports only one context. Abstraction may overfit wording or credit irrelevant tools. Holdout comparison measures reuse, and only strict gains replace an incumbent to prevent low-quality accumulation. Readable skills still need real execution; a local filter is not full evaluation.
Illustrative example: An audit skill that succeeds on its original example but fails on related unseen cases should not become the default.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Implementation details
Has RedEvoAgent locally red-teamed an actual product?
RedEvoAgent: Automatic Red-Teaming Agent with Experience-Driven Skill Evolution
Show answer and example
Answer: No. A safety mini-suite checks attribution and skill-update control flow. Replaying public artifacts is not a new external attack. Record skill provenance, evaluation splits, and replacement evidence without calling replay fresh execution.
Illustrative example: Loading an existing result confirms the protocol can be read, not that an attack just occurred.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Method comparison
How does attributing only decisive tools differ from crediting every tool in a successful trace?
RedEvoAgent: Automatic Red-Teaming Agent with Experience-Driven Skill Evolution
Show answer and example
Answer: Trace success does not mean each call contributed. Equal credit can fossilize irrelevant steps into skills. Decisive attribution retains experiences that change audit outcomes, reducing library noise. An attribution score does not establish a universal causal law.
Illustrative example: An irrelevant lookup before a decisive tool should not become a required skill simply because the trace succeeded.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Basics
Why does RedEvoAgent distill experience into skills rather than store whole traces?
RedEvoAgent: Automatic Red-Teaming Agent with Experience-Driven Skill Evolution
Show answer and example
Answer: Long traces contain noise and consume context. Readable abstract skills improve reuse, and tool credit focuses on decisive steps. Abstraction can lose detail, so skills require held-out validation before promotion.
Illustrative example: Extract 'verify permission boundaries first' from a long investigation rather than copying every irrelevant attempt.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Core mechanism
Why is behavior-aware paired validation better than a fixed validation-set average at finding local regressions?
Verify Smarter, Evolve Further: Efficient Harness Evolution through Behavior-Aware Verification
Show answer and example
Answer: A change affects only some behaviors. Unrelated tasks dilute differences and waste rollouts. Compare parent and child on affected tasks and require attribution to the change, distinguishing actual repair from noise. State the scope of a local pass; one behavior's improvement does not establish that every task improved.
Illustrative example: If only retry behavior changes, pair timeout and recovery tasks. Many easy successes could hide a retry loop in an overall mean.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Implementation details
Why is A_b required in addition to Δ_b>0?
Verify Smarter, Evolve Further: Efficient Harness Evolution through Behavior-Aware Verification
Show answer and example
Answer: A positive difference may come from random retries or environmental changes; A_b requires evidence attributable to the candidate. The local project has a real tool-feedback protocol without answer/guide but did not run the paper's four complete benchmarks. Local success cannot replace reported held-out gains, and structural labels do not prove causality.
Illustrative example: If two runs differ randomly and the candidate behavior was never checked for activation, do not credit the entire positive difference to the patch.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Method comparison
How does passing a local behavior acceptance gate differ from proving overall agent improvement?
Verify Smarter, Evolve Further: Efficient Harness Evolution through Behavior-Aware Verification
Show answer and example
Answer: The local gate requires positive paired gains on target behavior and evidence they are attributable, answering whether the particular change is worth accepting for that behavior. Overall capability includes uncovered tasks, other behaviors, and resource effects. Neither Δ_b nor A_b generalizes automatically. Retain the validation set and change scope rather than relabeling a local pass as broad validation.
Illustrative example: Improvements on five recovery cases can support limited acceptance. They do not show better code generation, mathematics, or all browsing tasks.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Basics
Why does HarnessLens treat validation cost as a bottleneck for harness search?
Verify Smarter, Evolve Further: Efficient Harness Evolution through Behavior-Aware Verification
Show answer and example
Answer: Fully running a fixed validation set for each candidate consumes many rollouts, and its mean can hide regressions in a few affected behaviors. HarnessLens focuses validation on behavior actually touched by a change, making evidence more targeted under a budget. State local coverage; saving evaluation does not justify skipping attribution or expanding quality claims.
Illustrative example: Rerunning every unrelated math problem after changing browser recovery can waste effort. Find matching behavior tasks, while noting that other behaviors have no new coverage.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Core mechanism
Why must a paper-to-code subplan read current repository state rather than following paper order alone?
DeepRepro: State-Aware Subplanning for Paper-to-Code Reproduction in Evolving Repositories
Show answer and example
Answer: Interfaces, dependencies, and files change during implementation. Prior assumptions can expire. Fine-grained subplans use current repository state and execution feedback to decide what to do next. The paper defines the goal; repository state defines what can run. Neither is replaced by one initial checklist.
Illustrative example: Example: The next paper step is evaluator integration, but the data interface has changed. Adapt to the new contract instead of generating scripts that no longer connect.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Implementation details
Does artifact replay count as rerunning official PaperBench Code-Dev?
DeepRepro: State-Aware Subplanning for Paper-to-Code Reproduction in Evolving Repositories
Show answer and example
Answer: No. Replay reads real exports from a pinned revision and checks plans, execution steps, and test coverage in paper and state order. It does not pay again for model API calls or the full implementation experiment. A perfect deterministic mini-suite verifies the state-driven path only, not official reproduction capability or cost.
Illustrative example: Example: Replaying an old test record shows whether a plan revision covered the failure; it does not regenerate code and run all tests in the current environment.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Method comparison
What can inspecting artifacts tell you, and what requires rerunning the reproduction?
DeepRepro: State-Aware Subplanning for Paper-to-Code Reproduction in Evolving Repositories
Show answer and example
Answer: Artifact inspection checks that code, logs, and tests correspond to the named paper and revision, auditing an existing execution. A rerun must actually install dependencies, call models, and run tests to see whether results reproduce now. The evidence scopes differ; a consistent local DeepRepro replay does not replace a full Code-Dev run.
Illustrative example: Example: A log shows that tests passed on one commit. Repeating the pass in a clean environment still requires installation and execution.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Basics
Why is paper reproduction an evolving software engineering task rather than a translation of formulas?
DeepRepro: State-Aware Subplanning for Paper-to-Code Reproduction in Evolving Repositories
Show answer and example
Answer: Equations are only part of the specification. Reproduction also involves dependencies, data formats, interfaces, and tests. Each step changes what can follow. DeepRepro studies how to implement and verify within an evolving repository while retaining alignment with the paper, not how to finish by writing one long plan.
Illustrative example: Example: The network is implemented, but loading data reveals a new axis-order requirement. The training code must fit that interface rather than blindly following section order.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why use semantic alignment and a masked listwise objective instead of simply making LightGCN vectors soft tokens?
When Memory Takes Gradients: Collaborative Vector Memory for Agentic Recommender Systems
Show answer and example
Answer: Matching projection dimension does not ensure use of collaborative geometry. Semantic alignment constrains the interface, and within-candidate ranking forces relevant memory to affect predictions. Candidate-driven retrieval changes history by option set. A frozen bank does not remove connector training or make explanations inherently faithful.
Illustrative example: Example: The same user needs different history for movies and books. Ranking supervision must teach soft tokens to affect the corresponding candidate scores.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
How does the mean candidate vector choose which collaborative memories to read from user history?
When Memory Takes Gradients: Collaborative Vector Memory for Agentic Recommender Systems
Show answer and example
Answer: Average the current candidates' vectors in the frozen bank, score historical items against that mean, select relevant items, and project them for the LLM. Changing candidates can change the read even when history is fixed. The local run verifies conditional retrieval, not training the paper's LightGCN, connector, or LoRA.
Illustrative example: Example: Switching the candidate set from cameras to hiking boots changes the mean and can surface photography versus outdoor interactions.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
How does collaborative soft-token memory differ from repeatedly rewriting a textual profile?
When Memory Takes Gradients: Collaborative Vector Memory for Agentic Recommender Systems
Show answer and example
Answer: Soft tokens preserve collaborative geometry from a frozen LightGCN bank and use projection and alignment to interface with an LM. Text profiles are readable but require generative rewrites and can lose catalog structure. Vector interfaces need training and interpretation limits. No extra text-maintenance calls does not mean zero inference cost.
Illustrative example: Example: Ranking can read collaborative vectors without asking an LLM to rewrite preferences, but projection, retrieval, and language forward passes still consume compute.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Why does CoVeMem retrieve collaborative memory based on the current candidate set?
When Memory Takes Gradients: Collaborative Vector Memory for Agentic Recommender Systems
Show answer and example
Answer: The average collaborative candidate representation describes the local need. The system retrieves matching history and projects it into LLM-readable soft tokens. This focuses memory on the ranking task rather than a fixed text profile of all history. New candidates can change retrieval for the same user.
Illustrative example: Example: Outdoor-gear candidates may retrieve related purchases, while skin-care candidates retrieve a different portion of the same user's history.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
Why must Complexity be calibrated to a specific learner instead of assigning a universal difficulty label?
What Makes Good Agentic Data? An ACE Lens on Data Generation for LLM Agents
Show answer and example
Answer: Learnability differs across models. Very easy examples add little signal and very hard ones may only generate failures. Accuracy first ensures credible samples, learner-relative complexity finds a useful band, and Diversity adds environment and behavior coverage. All three matter; length and tool count do not establish better data.
Illustrative example: A novice can learn from three-step tool tasks and fail every twenty-step one. A stronger model may need the latter, so the curriculum band should depend on the learner.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
How does public artifact evaluation prevent three-stage admission from becoming fictitious training results?
What Makes Good Agentic Data? An ACE Lens on Data Generation for LLM Agents
Show answer and example
Answer: Pin an actual dataset revision; record environments, questions, trajectories, verifier, and learner loss. Compare random and three-stage selection under equal budgets. The local path has admission contracts and artifact replay, not trained checkpoints or SOTA claims. Do not fill missing provenance or failures with synthetic high scores.
Illustrative example: A JSONL export that passes field checks is an auditable input. Without training and independent model evaluation, it cannot show an ACE success-rate gain.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
Why keep Complexity as learner challenge and Diversity as coverage separate?
What Makes Good Agentic Data? An ACE Lens on Data Generation for LLM Agents
Show answer and example
Answer: Complexity measures difficulty for the specified learner; Diversity measures differences among samples and coverage of task space. Diverse data can be easy and difficult data can repeat one pattern. Both require Accuracy: a novel but wrong trajectory is not good supervision, and static labels do not establish training benefit.
Illustrative example: Ten different tools used once each offer broad but easy coverage. Ten similar hard error cases challenge the learner but cover a narrow area.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why does ACE say more agent trajectories do not necessarily make better training data?
What Makes Good Agentic Data? An ACE Lens on Data Generation for LLM Agents
Show answer and example
Answer: Trajectories may be incorrect, too hard for the learner, or repetitive, and more data can amplify those issues. ACE selects for Accuracy, Complexity, and Diversity to account for quality and learner fit. Define and evaluate each empirically rather than inferring benefits from more steps or topics.
Illustrative example: A thousand copies of an incorrect tool call are not good data. Many simple tasks the learner has mastered may also teach little.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Core mechanism
Why norm-pool source history, compress target tokens, and add domain positions?
TransRetrieval: Scaling Up Transformer-Based Retrieval for Industrial Recommendation
Show answer and example
Answer: Norm weighting selects among noisy source events; target compression reduces cross-domain compute, while position/domain identity separates business meanings. Norm is not guaranteed relevant and compression can lose target detail.
Illustrative example: Mark article saves and product purchases as different while compressing both sides for ad retrieval.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Implementation details
Does 85% fewer target FLOPs imply equal end-to-end latency savings?
TransRetrieval: Scaling Up Transformer-Based Retrieval for Industrial Recommendation
Show answer and example
Answer: No. Structural counts omit memory movement and other service stages. No original 40B interactions, large ads catalog, or ANN are reproduced.
Illustrative example: Preprocessing or retrieval can dominate even if attention arithmetic falls.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Method comparison
How does norm weighting differ from averaging source tokens?
TransRetrieval: Scaling Up Transformer-Based Retrieval for Industrial Recommendation
Show answer and example
Answer: Larger norms get more weight, whereas mean is uniform. Norm is a proxy, not causal importance. Domain position encoding is not one shared constant.
Illustrative example: A high-amplitude noisy feature may receive too much influence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Basics
Why compress source history and target tokens?
TransRetrieval: Scaling Up Transformer-Based Retrieval for Industrial Recommendation
Show answer and example
Answer: Source noise and many target tokens both raise cross-attention cost. Norm-aware pooling and target compression act at different ends; domain embeddings limit confusion. Compression loses detail.
Illustrative example: Summarize source behavior and combine target fields before interaction.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Core mechanism
Why does DCEO use an actor to output contextual objective weights instead of one sitewide CTR/CVR/GMV mix?
DCEO: Direct Causal Effect Optimization for Long-Term User Value Modeling in E-commerce Search
Show answer and example
Answer: Fixed weights assume identical trade-offs across user states. The actor emits dynamic weights summing to one; the critic assesses relative intervention effects in training while serving uses the lightweight actor. Dynamic weights do not establish causal identification without credible controls.
Illustrative example: Exploration may matter for a new user and conversion for a mature shopper, but compare appropriate long-term feedback rather than just clicks.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Implementation details
Why does local DCEO run the held-out test only once for the validation champion?
DCEO: Direct Causal Effect Optimization for Long-Term User Value Modeling in E-commerce Search
Show answer and example
Answer: `causal_gain` and temperature define 20 candidate configurations evaluated only on fixed validation utility. Test only the selected champion to avoid using test variation as search feedback. Public proxy scores and the method's causal name are not evidence of real intervention effects.
Illustrative example: Testing all 20 and taking the highest converts test into validation. Lock the champion first and report any NDCG regression honestly.
Sections in the Chinese source: 本地复现. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Method comparison
What does DCEO's dynamic simplex weighting add over fixed log fusion, and what are its risks?
DCEO: Direct Causal Effect Optimization for Long-Term User Value Modeling in E-commerce Search
Show answer and example
Answer: Nonnegative weights summing to one can express different trade-offs per request. More flexibility can overfit or exploit proxy rewards, requiring validation and per-objective checks. Local Hit rose but NDCG fell slightly, so this is not a clean sweep.
Illustrative example: Pushing popular items forward can raise hits while degrading positions of relevant items.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Basics
What do DCEO's actor and user-level critic each do?
DCEO: Direct Causal Effect Optimization for Long-Term User Value Modeling in E-commerce Search
Show answer and example
Answer: The actor gives contextual weights to fuse click, conversion, and value scores immediately. The critic estimates the action's relative long-term effects during training. Serving keeps only the actor; that does not validate critic causality assumptions.
Illustrative example: The critic checks whether a shift from relevance toward transaction value yields incremental long-term benefit without running that evaluator at serving.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Core mechanism
Why separate vision, reasoning and instruction following instead of one score?
V-Rubrics: Visual Faithfulness via Rubric-Based Reinforcement Learning
Show answer and example
Answer: A visually correct reply can reason poorly; a coherent one may invent details. Atomic rubrics localize evidence to prefixes instead of obscuring it with terminal reward. Weights don't validate labels; local proxies aren't visual models.
Illustrative example: Correct formatting doesn't excuse a nonexistent object used as a premise.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Implementation details
Do fixed V-Rubrics scores establish actual visual judging?
V-Rubrics: Visual Faithfulness via Rubric-Based Reinforcement Learning
Show answer and example
Answer: No. Deterministic public candidates/plans test reward and credit without paper checkpoints, private data or full benchmarks. Fixed scores aren't vision.
Illustrative example: A supplied VF score tests aggregation, not seeing an image.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Method comparison
Can V-Rubrics dimension weights replace criterion coverage?
V-Rubrics: Visual Faithfulness via Rubric-Based Reinforcement Learning
Show answer and example
Answer: No. Weights combine detected dimensions; coverage determines which errors are seen. Raising VF cannot penalize errors none of its rubrics detect. Objective tradeoff and measurement reliability differ.
Illustrative example: Object-type-only checks may miss two becoming three regardless of VF weight.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Basics
What do VF, RC and IF measure in V-Rubrics?
V-Rubrics: Visual Faithfulness via Rubric-Based Reinforcement Learning
Show answer and example
Answer: Visual faithfulness, reasoning consistency and instruction following. They are separate axes, not candidates or ordered steps; identify error type before assigning reward and prefix credit.
Illustrative example: Wrong requested bullet count is IF; inventing an object is VF.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Core mechanism
Why learn fine-grained interpolation instead of one ratio for thinking and instruct weights?
GRIP: Granular Reward-Guided Parameter Interpolation for Efficient Reasoning
Show answer and example
Answer: Layers and parameters serve different capabilities, so a uniform mix can weaken reasoning and brevity. Reward-guided coefficients vary across weights. The local probability interpolation is a proxy, not the paper's weight-combination validation.
Illustrative example: Preserve computation habits while reducing redundant explanation by using different coefficients across parameter regions.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Implementation details
What shape and evaluation constraints apply to GRIP interpolation?
GRIP: Granular Reward-Guided Parameter Interpolation for Efficient Reasoning
Show answer and example
Answer: Weight tensors and coefficients must align at the stated granularity; broadcasting must match layer or parameter semantics. Rewards must follow the allowed data protocol. A deterministic candidate diagnostic does not prove paper-checkpoint reasoning or length gains.
Illustrative example: Do not broadcast layer coefficients as vocabulary-column coefficients merely because multiplication works.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Method comparison
How does GRIP interpolation differ from lowering inference temperature?
GRIP: Granular Reward-Guided Parameter Interpolation for Efficient Reasoning
Show answer and example
Answer: Weight interpolation changes distributions across contexts. Temperature rescales existing logits for one forward pass. Both can change length without representing the same mechanism or combining learned capabilities.
Illustrative example: Merging can change the favored reasoning step; lower temperature generally emphasizes already higher logits.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Basics
Does GRIP produce one merged model or run thinking and instruct models on every request?
GRIP: Granular Reward-Guided Parameter Interpolation for Efficient Reasoning
Show answer and example
Answer: It interpolates corresponding aligned parameters into one model and runs that model. It does not select experts online or generate two answers and vote. Arbitrary architectures cannot be combined without matching weights.
Illustrative example: Combine same-shape layer weights before one forward pass, not two responses followed by length selection.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
Why correct GRIN injection by the mixed distribution?
From Memorization to Absorption: Mixed-Policy RL for Continual Knowledge Injection
Show answer and example
Answer: An expert addition was not an unguided current-policy sample. Correct with the mixture denominator; reduce injection as capability improves and return to exploration. Gold must not appear in evaluation/deployment.
Illustrative example: Learn from an injected verified solution without logging it as a student exploration success.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
Why use q_mix rather than frozen reference in GRIN's importance ratio?
From Memorization to Absorption: Mixed-Policy RL for Continual Knowledge Injection
Show answer and example
Answer: q_mix describes actual sampling; reference anchors regularization. Track mixture origins/probabilities. The deterministic diagnostic does not reproduce the paper's knowledge-acquisition data.
Illustrative example: An old reference is not interchangeable with the expert that generated a sample.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
How does mixture correction differ from treating the expert answer as a student sample?
From Memorization to Absorption: Mixed-Policy RL for Continual Knowledge Injection
Show answer and example
Answer: Sampling includes the injection source, so the denominator must represent the mixture. Calling it pure on-policy misweights it and hides external knowledge.
Illustrative example: An inserted expert solution was not drawn solely from the student.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
Why inject a golden response only into GRIN failure groups?
From Memorization to Absorption: Mixed-Policy RL for Continual Knowledge Injection
Show answer and example
Answer: All-fail groups lack positive contrast. Verified solutions supply direction; preserve more native exploration when the student succeeds. Gold is training supervision, not evaluation input.
Illustrative example: Add a verified answer only after all attempts fail, not to every prompt.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Core mechanism
Why query a clue-conditioned teacher on the student's own prefixes?
Where to Look Matters: On-Policy Self-Distillation for Long-Video Understanding
Show answer and example
Answer: Actual student prefixes reduce trajectory mismatch while clue segments focus evidence. Clues are training privilege; at inference the student must handle full video without markers, answers or teacher.
Illustrative example: Guide correction of an incomplete event ordering during training, without giving the manual clip at deployment.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Implementation details
Why score Clue-OPSD KL on student rather than teacher prefixes?
Where to Look Matters: On-Policy Self-Distillation for Long-Video Understanding
Show answer and example
Answer: Guide actually visited states with aligned token positions despite different visual conditions. The mini-suite does not execute a full video-checkpoint experiment.
Illustrative example: If the student misidentifies a person, teacher must rate that prefix, not its own ideal opening.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Method comparison
How does a clue teacher differ from shortening the student's video?
Where to Look Matters: On-Policy Self-Distillation for Long-Video Understanding
Show answer and example
Answer: Shortening input simplifies the task; teacher-only privilege changes supervision while preserving full-video difficulty and costs.
Illustrative example: Seeing a solution clip and seeing all footage are not matched evaluations.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Basics
Why are relevant time intervals privileged training information in Clue-OPSD?
Where to Look Matters: On-Policy Self-Distillation for Long-Video Understanding
Show answer and example
Answer: A focused teacher can guide evidence retrieval in long video. Student still receives full video/question and loses clue and teacher at deployment.
Illustrative example: Teacher knows the opening-door timestamp; deployed student must find it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Core mechanism
Why prefer an executable scorer to a general VLM judge for some spatiotemporal constraints?
VBVR-Pro: A Scalable and Verifiable Suite for Native Visual Reasoning
Show answer and example
Answer: Explicit intermediate states, constraints, and goals can be checked per example rather than by fluency. Executable rewards cover only defined rules. Scorer correctness, gold isolation, and actual trajectory generation still need separate checks.
Illustrative example: Verify each move avoids walls and reaches the target; a fluent explanation does not excuse crossing a wall.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Implementation details
What does the local programmatic reward's gain over a noisy baseline to a perfect score show?
VBVR-Pro: A Scalable and Verifiable Suite for Native Visual Reasoning
Show answer and example
Answer: It shows deterministic task scoring and failure attribution on the supplied programs are more stable than a noisy judge analogue. It does not show a multimodal generator gained ability after RLVR. Real trajectories, model updates, and independent evaluation are still needed.
Illustrative example: Scoring a known valid trajectory as one tests the scorer, not a generator's ability to discover it.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Method comparison
Is a weighted constraint reward equivalent to requiring all constraints to pass?
VBVR-Pro: A Scalable and Verifiable Suite for Native Visual Reasoning
Show answer and example
Answer: No. Weighted sums permit compensation; a hard conjunction makes any violation infeasible. Higher total reward does not guarantee every constraint. This compares objective semantics, not an additional author ablation.
Illustrative example: One of two equally weighted checks gives half credit, while a mandatory safety check cannot be offset by success elsewhere.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Basics
Why can an attractive final image fail the visual reasoning task?
VBVR-Pro: A Scalable and Verifiable Suite for Native Visual Reasoning
Show answer and example
Answer: Rules may constrain motion, intermediate states, and forbidden areas. A correct-looking ending can hide process violations. VBVR-Pro turns known rules into executable checks, without measuring undefined quality or assuming every task has a reliable scorer.
Illustrative example: An object can reach its goal by crossing a forbidden area; only the trajectory check catches it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Core mechanism
Why can CKA distill structure when teacher and student feature widths differ?
MLLMCLIP: Feature-Level Distillation of MLLM for Robust Vision-Language Representations
Show answer and example
Answer: CKA aligns relationships between examples without coordinate-by-coordinate equality. The teacher selects informative tokens by attention; CKA constrains student image and text representations. The deployed student stays dual-tower and does not reproduce the teacher's entire reasoning.
Illustrative example: A 1,000-dimensional teacher and 500-dimensional student can both put similar examples close without matching coordinates.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Implementation details
Does the real-checkpoint path train the full CLIP student?
MLLMCLIP: Feature-Level Distillation of MLLM for Robust Vision-Language Representations
Show answer and example
Answer: No. It freezes SmolVLM2 and the CLIP encoders, trains a dimension-matching feature projection, and selects weights on separate validation. Held-out CKA and neighbor overlap do not improve consistently across three seeds. A perfect score caused by degenerate labels was invalidated.
Illustrative example: Running CKA after projection checks the operator path, not full student training over the paper's 26 benchmarks.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Method comparison
How does attention token selection change MLLMCLIP supervision relative to distilling all tokens?
MLLMCLIP: Feature-Level Distillation of MLLM for Robust Vision-Language Representations
Show answer and example
Answer: It builds structural supervision from selected informative teacher positions across layers rather than every position. This changes information selection, not CKA's ability to handle different widths. Selection can miss useful details, so superiority requires a controlled comparison.
Illustrative example: Select a subset of locations when a teacher layer includes many background tokens. More tokens do not necessarily give the student more useful knowledge.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Basics
Why use a multimodal LLM teacher but keep a dual-tower MLLMCLIP student?
MLLMCLIP: Feature-Level Distillation of MLLM for Robust Vision-Language Representations
Show answer and example
Answer: The teacher supplies rich cross-modal semantics but is expensive to deploy as a generator. The student distills relationships while encoding images and text independently. Teacher training cost and online retrieval cost are separate.
Illustrative example: Distill offline, cache item image embeddings, and retrieve with a text vector without generating an answer for each item.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Core mechanism
Why do versions, action intent, and edit size teach planning patterns better than score curves alone?
TraceML: An Empirical Analysis of Human-Agent Planning in Machine Learning Development
Show answer and example
Answer: Scores show outcomes but cannot distinguish architecture exploration, tuning, or returning to an old approach. Linking edits to intent and effect reveals stage changes and reopened approaches for a prior. Patterns in human traces still need agent validation and are not optimal by default.
Illustrative example: Example: Two score drops can come from trying a feature or accidentally deleting configuration. Version and action records tell whether to continue or revert.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Implementation details
Can TraceML's failure-reflection prior keep reading hidden routes on validation or test?
TraceML: An Empirical Analysis of Human-Agent Planning in Machine Learning Development
Show answer and example
Answer: No. Learn the prior on train, then freeze it and act on actual tool feedback without hidden answers or routes. PlanBench stage and reopen counts are mechanism diagnostics, and ToolRoute is not the full Kaggle experiment. They are different evidence.
Illustrative example: Example: Adjust after a public test error, but do not reopen a solution by reading the reference plan.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Method comparison
How do human stage alternation and revisiting old ideas differ from a monotonic plan?
TraceML: An Empirical Analysis of Human-Agent Planning in Machine Learning Development
Show answer and example
Answer: Humans move among analysis, implementation, and evaluation and revisit hypotheses after new evidence. A monotonic plan can discard abandoned paths permanently. A prior can encourage useful revisits, not enforce cycles on every task. Closer behavior does not guarantee better scores.
Illustrative example: Example: New validation identifies a data problem, making a previously abandoned simple model worth reconsidering instead of increasing complexity.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Basics
Why does TraceML record the whole human ML development process rather than just the highest-scoring model?
TraceML: An Empirical Analysis of Human-Agent Planning in Machine Learning Development
Show answer and example
Answer: Final code and scores hide exploration, abandonment, revisiting, and budget allocation. TraceML records decisions and versions as planning evidence. Observed human behavior is not proof of optimality; downstream use requires independent tasks.
Illustrative example: Example: A small final model conceals the failed larger-model experiment that identified the bottleneck and motivated the pivot.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Core mechanism
Why not optimize shared-cache scheduling for prefix hit rate alone?
TOPAS: Workflow-Aware Prefix-State Scheduling for Multi-Agent LLM Serving
Show answer and example
Answer: Hits improve local reuse but can delay critical workflow paths and incur expensive transfers or preemption. TOPAS combines longest remaining path, downstream reuse, and movement cost, with aging against starvation. Hit rate is an intermediate measure, not whole-task completion time.
Illustrative example: Example: Serving many high-hit small requests can block a nearly finished main workflow. Sacrificing hits to unblock the critical path can deliver sooner.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Implementation details
Do simulated prefix hits and ToolRoute results prove real LLM serving speedup?
TOPAS: Workflow-Aware Prefix-State Scheduling for Multi-Agent LLM Serving
Show answer and example
Answer: No. The mini-suite simulates prefix reuse, critical path, and aging. Stateful tools do not exercise real KV serving kernels. Paper mean/p99 JCT comes from real system experiments requiring requests, memory, movement, and wall-clock timing. Proxy costs and hit counts are not convertible directly.
Illustrative example: Example: Three extra hits do not save three identical prefill intervals. Prefix lengths and scheduling overhead matter.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Method comparison
What latency risks do critical-path priority and aging address?
TOPAS: Workflow-Aware Prefix-State Scheduling for Multi-Agent LLM Serving
Show answer and example
Answer: Critical-path priority identifies requests delaying whole workflows. Aging raises priority with wait time so tasks are not repeatedly starved. These address dependencies and fairness, alongside cache reuse and movement. No single priority makes every task fastest.
Illustrative example: Example: Aging eventually helps a short chain skipped for low cache reuse, while a long chain's blocked node still matters for critical-path latency.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Basics
Why does TOPAS consider the workflow's longest remaining path rather than only individual progress?
TOPAS: Workflow-Aware Prefix-State Scheduling for Multi-Agent LLM Serving
Show answer and example
Answer: One request can block downstream work, so local priority can increase tail waiting. Critical-path estimates include dependencies and remaining work, then trade them against prefix reuse, transfer, and preemption. Optimize completion time, not cache hits alone.
Illustrative example: Example: A short request unblocks five later nodes and may deserve priority over a long request that has already run extensively.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why isn't SKILL.state the full conversation with an extra state block?
SKILL.state: Scalable Long-Horizon Agent Skills
Show answer and example
Answer: The next step receives only skill instructions, current state, and latest observation. Older conversation is not replayed, so persistent information must be written explicitly. Recursive patches preserve untouched fields and allow deletion; validation precedes commit. Full history would bypass state sufficiency and the capacity constraint.
Illustrative example: Example: Save a previously confirmed file path in state to use it next turn. Secretly keeping the full chat would not test whether state suffices.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
If action execution fails after valid state is committed, does the runtime undo state and external side effects?
SKILL.state: Scalable Long-Horizon Agent Skills
Show answer and example
Answer: No. The paper commits valid state before the action. A local executor error does not trigger blind retry or imply external rollback. The caller must process the failure as latest observation. Format-retry and character caps are local safeguards; counter events do not prove a real LLM can generate correct patches.
Illustrative example: Example: After recording a payment attempt, a network timeout does not justify resending it. Check the actual outcome to avoid double payment.
Sections in the Chinese source: 核心公式, 本地复现, 成本与复现边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
How does a fixed-schema state update differ from a generic rolling summary?
SKILL.state: Scalable Long-Horizon Agent Skills
Show answer and example
Answer: Recursive patches preserve untouched fields and use null for explicit deletion. The runtime can check field names and types. Natural-language summaries are flexible but harder to constrain for overwrite and deletion. Valid structure does not guarantee factual contents or correct model updates.
Illustrative example: Example: A patch changing progress must preserve order_id. A prose rewrite may lose that ID without a clear field-level rejection.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Why not rely on the model to recall current progress from all chat history in a long task?
SKILL.state: Scalable Long-Horizon Agent Skills
Show answer and example
Answer: History grows and contains many failed attempts. Continuation needs clear completed work, unmet requirements, and key resources. SKILL.state maintains them as constrained execution state so new actions use current facts instead of reconstructing the whole past. The schema must be expressive enough.
Illustrative example: Example: After changing three files, one failing test remains. The next turn should see that explicit item instead of guessing from dozens of messages.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Core mechanism
Why predict marginal progress for routing instead of comparing average model accuracy?
Show answer and example
Answer: A stronger model may offer little progress in one state but unblock another. Marginal progress uses completion, trend, and quality to value the next call against time and long-run budget. Average accuracy misses stage differences, and predictions still require execution feedback.
Illustrative example: Example: A cheap model can repair formatting while a difficult proof needs stronger reasoning. Calling the average best on every step wastes budget.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Implementation details
Does the local frozen program router have the same scope as the paper's multi-model progress predictor?
Show answer and example
Answer: No. Local work compiles verified tool choices into routes, freezes them for validation and test, and executes. Mini-suite checks cover progress, meta-gating, and downgrade, not paper models, serving resources, or full benchmarks. Proxy costs and program selection are not measured joules or citation precision.
Illustrative example: Example: Reading two fewer candidates does not imply the same energy saving on HumanEval without actual model and hardware measurements.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Method comparison
How does a quality, time, and long-run-cost meta-gate compare with always choosing the cheapest model?
Show answer and example
Answer: Cheap calls can accumulate retries and cost more overall. A meta-gate trades predicted progress, call price, and budget pressure, sometimes paying more for a better candidate. Report delivery quality, time, and total budget instead of just per-call cost.
Illustrative example: Example: Three cheap calls stall while one expensive call solves it. If the expensive call also stalls, execution must override the prediction.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Basics
Why does ProgRouter track changing model needs during one task?
Show answer and example
Answer: Reasoning requirements vary by stage. Always using a strong model is costly; a weak model can stall. ProgRouter routes each next call by current progress. Static rankings do not quantify help now, and real gains depend on time and call budgets.
Illustrative example: Example: A cheap model locates a file, then a stronger model handles complex inference. Overall rank alone does not justify using it for both.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Core mechanism
Why separate memory, planning, action protocol, and tools when generating a harness per task?
JIT-Agent: Scaling Harness Intelligence via Just-in-Time Harness Evolution
Show answer and example
Answer: Task bottlenecks vary by layer. Modules make proposals and repairs target the right object and let configuration archives yield transferable patterns. Just-in-time generation still needs validation; each module must meet its execution contract. Editing prompt text without generating matching tools or state interfaces does not implement all four modules.
Illustrative example: A long task may need new memory, whereas short arithmetic needs only a calculation tool. Do not rebuild everything or mix incompatible protocols.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Implementation details
Why separate local training-time route repairs from frozen test execution?
JIT-Agent: Scaling Harness Intelligence via Just-in-Time Harness Evolution
Show answer and example
Answer: Training tasks can write repair experience. After freezing, validation and test use only public observations, tool metadata, and real feedback, not answers or reference plans. Report low controlled success rates and real ToolRoute results under their own protocols. Without the original model and full DeepSearchQA, do not import author scores.
Illustrative example: A test error may be handled using a frozen recovery rule. Adding the hidden correct route to the archive and then calling it successful adaptation is invalid.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Method comparison
What are the trade-offs between on-demand generation and reusing one fixed harness?
JIT-Agent: Scaling Harness Intelligence via Just-in-Time Harness Evolution
Show answer and example
Answer: On-demand configuration fits diverse tasks, and an archive can help transfer. Generation and repair add development cost and possible instability. A fixed harness is easier to maintain but can be overly generic. Count candidate generation and actual failures rather than cherry-picking the final successful configuration and ignoring exploration cost.
Illustrative example: Long retrieval may need citation checking, whereas simple calculation may not. Task-specific generation distinguishes them, but an unnecessary checker may slow the easy task.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Basics
Why does Jit-Agent generate a workflow for the current task rather than choose only a fixed prompt?
JIT-Agent: Scaling Harness Intelligence via Just-in-Time Harness Evolution
Show answer and example
Answer: Tasks need different memory, planning, action protocols, and tools. A universal harness can be heavy or lack key support. Jit-Agent composes a workflow when a task arrives so execution organization matches the request. Instant generation still needs quality constraints and frozen evaluation. Complete modules do not guarantee an advantage over fixed workflows.
Illustrative example: One-step arithmetic may not need long-term memory, whereas organizing several pages can need evidence collection and progress tracking. Task features should guide which modules are used.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why calibrate semantic and I/O similarity edges with remove, substitute, and reorder probes?
CaSKG: Counterfactual-Causal Skill Graphs for Scalable Agent Skill Retrieval
Show answer and example
Answer: Similarity helps recall but does not prove a dependency or order. Text counterfactuals alter local structure to see whether the relation still holds; smoothing retains more reliable edges. This is evidence under the probes, not a verified causal effect in a real environment.
Illustrative example: Example: Two skills mention uploading files, but removing authentication exposes a prerequisite. Shared vocabulary alone does not make them interchangeable.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
What do s, n, and the prior contribute to Bayesian smoothing?
CaSKG: Counterfactual-Causal Skill Graphs for Scalable Agent Skill Retrieval
Show answer and example
Answer: The posterior edge score is (α0+s)/(α0+β0+n). Successes add support, probe count controls shrinkage, and the prior avoids small-sample extremes. Publish only edges above threshold. One success is not probability one, and isolated local ToolRoute is not the full paper environment.
Illustrative example: Example: One probe success and a hundred successes are not equivalent once the prior and different evidence amounts are considered.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
How does evidence from text counterfactual probes differ from real environmental interventions?
CaSKG: Counterfactual-Causal Skill Graphs for Scalable Agent Skill Retrieval
Show answer and example
Answer: Text probes edit skill descriptions or order to calibrate relations, but may reflect judgments about wording. Real execution must verify action consequences and state. A causal name does not turn textual edge reliability into unconfounded causal evidence; distinguish mechanisms from capability.
Illustrative example: Example: A model predicts deletion of a step causes failure, but another tool may fill the same role. Test execution rather than rely on the prediction.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Why are several similar skills insufficient for an executable plan when the library grows?
CaSKG: Counterfactual-Causal Skill Graphs for Scalable Agent Skill Retrieval
Show answer and example
Answer: Skills can have order, input/output, and substitution relations. Similar wording does not make them compose. CaSKG retrieves skills together with reliable relations to avoid related but incompatible candidates. Calibration is needed beyond text similarity to claim dependency.
Illustrative example: Example: Both lookup and modification relate to orders, but modification may require confirming payment first. Omitting that relationship can cause a bad sequence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Core mechanism
Why separate tool necessity from fallback reflection on unreliable evidence?
AdaVDR: Adaptive Tool Use and Reflection for Video Deep Research
Show answer and example
Answer: Necessity decides whether to gather external evidence; reflection decides whether existing evidence needs another check. Capability-based filtering saves unnecessary calls and redundancy penalties guide training. Ignoring unreliable evidence merely to call less can hurt video research, so minimum call count is not optimal by itself.
Illustrative example: Example: Do not search to confirm visible running. If identity clues disagree, recheck rather than keep the first conclusion.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Implementation details
How should local ToolRoute success be reported alongside cost and video scope?
AdaVDR: Adaptive Tool Use and Reflection for Video Deep Research
Show answer and example
Answer: Retain added verifier cost and state that local runs use an oracle-free stateful tool environment, not the paper's video data or full VDR system. Perfect PlanBench is a mechanism diagnostic. Fallback counts or success under another protocol are not video research gains.
Illustrative example: Example: Report that an extra verification step bought local success rather than claim higher success and fewer video calls from the same result.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Method comparison
How does a redundancy-aware reward differ from penalizing every call?
AdaVDR: Adaptive Tool Use and Reflection for Video Deep Research
Show answer and example
Answer: Redundancy penalties target unnecessary or repeated calls while preserving useful evidence gathering. A flat penalty can deter necessary tools. Evaluate answer reward and reflection as well, since recovery and verification may require justified extra cost.
Illustrative example: Example: Repeating a search with no new information can be redundant, while checking conflicting sources may be essential.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Basics
Why judge tool necessity relative to the target model's capabilities?
AdaVDR: Adaptive Tool Use and Reflection for Video Deep Research
Show answer and example
Answer: Do not repeat external calls for things the model already handles; use tools for missing information. Necessity depends on model and evidence, not an unconditional prohibition. Reflect when intermediate evidence is unreliable rather than minimize calls at the cost of needed checks.
Illustrative example: Example: Clear colors can be read directly, but small text may need zoom or retrieval. A weaker vision model changes that boundary.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why separate live adaptation into code allocation, interest timescales, and intermittent policy updates?
TAGR: Temporally Adaptive Generative Recommendation for Industrial Live-Streaming Advertising
Show answer and example
Answer: Item content, users, and policies drift differently. LSID refreshes assignments behind a stable vocabulary, IAG fuses temporal interests, and IOPO runs NTP between on-policy bursts. Bounded local gates are not an online GRPO loop.
Illustrative example: A changed product, shifted intent, and stale policy need separate fixes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Does local TAGR run real GRPO bursts and intermittent NTP?
TAGR: Temporally Adaptive Generative Recommendation for Industrial Live-Streaming Advertising
Show answer and example
Answer: No live bursts or production value labels; only bounded adaptation and failure cases on static public data. Real scheduling records sampling policy, both rewards, and maintenance versions.
Illustrative example: Adding static losses does not execute a timed online GRPO update.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
Is stable LSID vocabulary the same as freezing item-to-code assignments?
TAGR: Temporally Adaptive Generative Recommendation for Industrial Live-Streaming Advertising
Show answer and example
Answer: No. Token symbols can stay stable while assignments refresh. Freezing mappings also preserves path identities. Compatibility depends on which changes.
Illustrative example: The same digits can encode a stream on a new path.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Why combine LSID, IAG, and IOPO adaptation?
TAGR: Temporally Adaptive Generative Recommendation for Industrial Live-Streaming Advertising
Show answer and example
Answer: Dynamic codes address live content, multiscale history captures intent, and intermittent online optimization addresses reward-policy drift. One component cannot cover all three.
Illustrative example: Product changes, immediate user intent, and bidding value affect different paths.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Core mechanism
Why retain only nonnegative teacher signal for success and nonpositive for failure?
OPDVR: On-Policy Distillation with Verifiable Rewards
Show answer and example
Answer: Token ratios can conflict with terminal outcome. One-sided ReLU uses verifiable correctness to gate implicit reward, not add a standalone RL loss. Coarse labels don't prove each token and remain scoring-only.
Illustrative example: Suppress a negative imitation signal for a correct alternative without requiring teacher wording.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Implementation details
Which half-axis does OPDVR retain for correct and wrong trajectories?
OPDVR: On-Policy Distillation with Verifiable Rewards
Show answer and example
Answer: Correct keeps nonnegative log ratios; wrong keeps nonpositive; zero stays zero. Verifier is a fixed training signal, not rollout input. Local candidates aren't Qwen math rollouts.
Illustrative example: A positive log ratio on a failure becomes zero, not a sign-flipped extra penalty.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Method comparison
How does OPDVR differ from adding another RL loss?
OPDVR: On-Policy Distillation with Verifiable Rewards
Show answer and example
Answer: It removes the inconsistent part of implicit reward rather than mixing two losses with compensating weights. The objectives differ.
Illustrative example: Zero out positive teacher reward on an error rather than add a negative term hoping it cancels.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Basics
Why use a verifier to correct OPDVR teacher reward signs?
OPDVR: On-Policy Distillation with Verifiable Rewards
Show answer and example
Answer: Teacher/student log ratios can oppose trajectory correctness. Retain only sign-consistent credit to avoid reinforcing errors or suppressing correct trajectories.
Illustrative example: Do not punish a different correct solution just because the teacher dislikes it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Core mechanism
Why distill from a frozen general teacher before continuing agent RL?
OPDSearch+: Search-Enhanced On-Policy Distillation with Reinforcement Learning
Show answer and example
Answer: Dense guidance on student search traces improves the starting behavior. Verifiable RL then adapts to environmental results rather than remain bound to teacher preferences. No specialized teacher training required; bias and proxy limits remain.
Illustrative example: Learn to ask workable queries, then let retrieved evidence and final verification adjust habits.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Implementation details
Does local OPDSearch+ execute real query, reasoning and answer tokens?
OPDSearch+: Search-Enhanced On-Policy Distillation with Reinforcement Learning
Show answer and example
Answer: No. Candidate-level process axes proxy generation/search feedback without real index/model. The real mask covers only generated text, never returned documents.
Illustrative example: A process score doesn't mean a Wikipedia search ran.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Method comparison
How does on-student search distillation differ from offline teacher demonstrations?
OPDSearch+: Search-Enhanced On-Policy Distillation with Reinforcement Learning
Show answer and example
Answer: The teacher sees states and prefixes the student actually reached, including mistakes; offline continuations come from teacher states. Different recovery coverage isn't a universal performance ranking.
Illustrative example: Online scoring sees a wrong first query that a correct teacher demonstration never visits.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Basics
Should OPDSearch+ students imitate search-result text as their own output?
OPDSearch+: Search-Enhanced On-Policy Distillation with Reinforcement Learning
Show answer and example
Answer: No. Students generate queries, reasoning and answers; search observations condition later actions. Teacher guidance doesn't make tool output a student target.
Illustrative example: Use a webpage excerpt to ask the next query rather than role-play the returned page verbatim.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Core mechanism
Why align at scale before curated relevance and cross-scale distillation in WeMM?
WeMM-Embedding: WeChat Multi-Modal Embedding Technical Report
Show answer and example
Answer: Initial alignment builds a shared space; refined relevance addresses items that look similar but fail the query. Distillation and Matryoshka representations carry semantics across sizes. Closeness alone is not business relevance or correctness.
Illustrative example: Red shoes match “red items,” but “shoes for rain” needs relevance supervision beyond appearance.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Implementation details
Can local MovieLens Matryoshka results replace WeChat A/B?
WeMM-Embedding: WeChat Multi-Modal Embedding Technical Report
Show answer and example
Answer: No. Public content and collaborative views test shared space, refinement, and truncation without the original checkpoint or private multimodal pairs. MMEB and WeChat results belong to the authors' system. Local low-dimensional proxies cannot reproduce business effects or rank on the same table.
Illustrative example: Compare Recall for four- and 16-dimensional proxy vectors locally; do not project it to video-feed percentage gains.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Method comparison
How does WeMM cross-scale KD differ from supervising only the largest embedding?
WeMM-Embedding: WeChat Multi-Modal Embedding Technical Report
Show answer and example
Answer: It sums teacher-to-student KL at designated Matryoshka sizes, so smaller outputs receive explicit signals. A common semantic target does not guarantee identical quality or arbitrary truncation. Compare retrieval quality and cost at specified sizes.
Illustrative example: Aligning the full vector does not guarantee the first half works well; the multiscale loss explicitly supervises that distribution.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Basics
How does interleaved WeMM input differ from a single image vector?
WeMM-Embedding: WeChat Multi-Modal Embedding Technical Report
Show answer and example
Answer: One input can combine text, images, video, and document segments in order before encoding. It represents combinations rather than indexing each modality in isolation. A shared space does not remove the need to train and validate combinations.
Illustrative example: A product photo plus “same design in blue” differs from either the photo or the color term alone.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
Why weight advantage standardization by action-token count for a token-mean actor loss?
Show answer and example
Answer: Equal trajectory votes during whitening conflict with token-mean loss and change how long and short rollouts influence updates. Token weighting aligns normalization with the optimization measure. Freezing the baseline at dispatch prevents completion-order leakage; it is separate from redefining rewards by text length.
Illustrative example: A 100-token trace occupies more loss positions than a 10-token trace. Giving them one vote each in normalization would mismatch actual gradient averaging.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
Do local event-time and token-measure checks include asynchronous Qwen training?
Show answer and example
Answer: No. A deterministic mini-suite checks frozen values, history timing, and action-token weighting, without real asynchronous models, ALFWorld, or Math-TIR. Event counts verify branches, not paper reward-curve area or throughput gains.
Illustrative example: Out-of-order completion of ten fixtures tests timing, not real worker latency, policy staleness, or parameter updates.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
What changes between freezing a baseline at dispatch and using the latest baseline at completion?
Show answer and example
Answer: Freezing associates each rollout with prompt value visible when dispatched. The newest value at completion incorporates later arrivals, affecting rollouts differently by duration. SPO++ indexes evidence by generation-policy events, so later history cannot override frozen conditions. This correction is independent of token-weighted standardization.
Illustrative example: A goes out before B but finishes after it. B can update future baselines without retroactively entering A's dispatch-time history.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why can asynchronous agent RL depend on which task finishes first?
Show answer and example
Answer: Long tasks have variable durations, so asynchronous training uses results as they arrive. Completion order can correlate with difficulty and policy version. SPO++ keeps throughput while aligning advantage baselines and statistics with the correct generation events. Fast completion is not representativeness; track sampling-time conditions rather than backfilling from later outcomes.
Illustrative example: Easy tasks may return early and change historical values. A hard task dispatched earlier should not be treated as sampled with that new baseline.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Core mechanism
Why jointly train tool creation and use in one policy?
SMITH: Self-Improving Tool-Using Agents through Multi-Aspect Verification
Show answer and example
Answer: Rewarding runnable code alone can produce awkward tools or tools that fail new tasks. Training only use cannot improve design. Mixed build and held-out use connect schema, execution, and outcome signals to encourage reusable abstractions. Passing examples is not independent problem solving.
Illustrative example: Example: A tool hard-codes a training constant and earns execution reward, but held-out calls expose the lack of reuse.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Implementation details
Is the local structured-plan tool equivalent to a Python execution sandbox?
SMITH: Self-Improving Tool-Using Agents through Multi-Aspect Verification
Show answer and example
Answer: No. Local work separates build and use and runs three verifiers, but uses controlled plans without Qwen3-4B, a Python sandbox, or Reasoning-Gym. Generation and reuse counts verify control flow, not paper code-tool training effects.
Illustrative example: Example: A rule interpreter can execute an add node without safely compiling or running arbitrary model-generated Python.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Method comparison
Why aren't schema, code execution, and final answer rewards interchangeable?
SMITH: Self-Improving Tool-Using Agents through Multi-Aspect Verification
Show answer and example
Answer: A schema only needs to parse. Code execution only needs to run without crashing. The final result tests whether the task was solved. Separate signals expose different faults. Harder problems should encourage abstraction, not more hard-coded examples passed off as generalization.
Illustrative example: Example: A function has a valid signature and returns a number using the wrong formula. It passes the first two checks but fails the answer check.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Basics
Why does SMITH learn to create reusable tools rather than only use existing ones?
SMITH: Self-Improving Tool-Using Agents through Multi-Aspect Verification
Show answer and example
Answer: Fixed tools may not cover new tasks, and rewriting one-off code wastes effort. Building and later using tools links interface and implementation to real use. Code generation does not establish usefulness; format, execution, and downstream answers can fail independently.
Illustrative example: Example: Repeated cleaning needs can justify a dedicated tool. Keep it only if later tasks can call it correctly and finish cleaning.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why activate, revise, or deduplicate skill candidates instead of appending every success permanently?
SkillForge: Automated Skill Discovery and Refinement for Tool-Using Agents
Show answer and example
Answer: One success may depend on incidental conditions. Old skills can go stale or conflict. Multi-path induction proposes candidates, then policy choices and environmental feedback determine use and revisions. Appending expands the library without ensuring fit and can reinforce bad retrievals.
Illustrative example: Example: A download skill hard-codes an old button. If it fails in a new UI, revise or deactivate it instead of stacking another similar document.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
What must the local check record for a_t=(environment action, skill choice)?
SkillForge: Automated Skill Discovery and Refinement for Tool-Using Agents
Show answer and example
Answer: Record the skill selected by the policy at action time and its execution outcome; a post-hoc label is not enough. Local structured plans verify invocation, posterior, and revision control flow without LLM teacher induction or RL. Reuse counts are not a learned policy or paper environment score.
Illustrative example: Example: Selecting a similar skill after an action to explain it is not actual use. Selection must influence action generation.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
How does multi-path induction from success, failure, and contrasts differ from summarizing successes only?
SkillForge: Automated Skill Discovery and Refinement for Tool-Using Agents
Show answer and example
Answer: Success supplies usable steps, failure exposes boundaries, and contrast helps identify meaningful differences. Candidates need not repeat one success, but remain hypotheses until environment validation. Do not convert guesses in failure narratives into reliable rules or generalize from one successful case.
Illustrative example: Example: Both trajectories use the same query, but only the successful one verifies results. Propose verification as important and test it rather than immediately claim causality.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Why does SkillForge teach skill selection rather than merely supplying a catalog?
SkillForge: Automated Skill Discovery and Refinement for Tool-Using Agents
Show answer and example
Answer: Procedural knowledge does not guarantee selection at the right time. Skills can be ignored, misused, or mismatched to current state. SkillForge learns invocation with actions and improves contents from execution, focusing on cooperation between knowledge and policy rather than document count.
Illustrative example: Example: A stored validate-before-import skill does not help an agent that always imports directly. Task feedback must affect selection.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
Why give a corrective hint only to the teacher on critical error steps instead of sharing full answers at every step?
AHEAD: Agentic Hints for Effective Agent Development
Show answer and example
Answer: Ordinary steps already get environment feedback. Blanket privileged hints waste signal and hide where correction matters. After locating an error, compare log probabilities for the same generated action with and without the hint, then adjust terminal advantage within bounds. Hints stay on the training teacher side and cannot enter deployment student context.
Illustrative example: If only the third step chooses the wrong tool, target that choice. Letting every step see the final solution is not autonomous completion.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
How do deterministic local error locations, real ToolRoute, and the paper's LLM analyzer differ?
AHEAD: Agentic Hints for Effective Agent Development
Show answer and example
Answer: The mini-suite uses deterministic positions to test hint routing, with no LLM analyzer, Qwen GRPO, or three original benchmarks. ToolRoute is a different isolated feedback protocol, not complete algorithm training. Hints, gold, and reference actions stay in permitted training contexts and are not visible to the test policy.
Illustrative example: A fixture that identifies step six as wrong tests gating, not autonomous localization of that step in a real long failure.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
How does multiplying the original episode advantage differ from using the teacher gap as a new reward?
AHEAD: Agentic Hints for Effective Agent Development
Show answer and example
Answer: Multiplication modulates the original advantage within clip and λ limits, preserving the terminal signal. Replacement would turn teacher preferences into a separate objective. The gap is a likelihood difference for the same token in two contexts, not success probability or causal contribution.
Illustrative example: A hint's stronger support for a token in a failed trajectory does not unconditionally reward it as success; original advantage and the bounded factor still apply.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why does AHEAD want more than one success or failure signal across an entire long-horizon agent trajectory?
AHEAD: Agentic Hints for Effective Agent Development
Show answer and example
Answer: A few critical errors may determine the outcome, while other steps are reasonable or merely affected by them. Uniform credit cannot identify what to fix. AHEAD uses targeted teacher signals to adjust learning on those steps. Fine-grained credit depends on error localization, and privileged teacher information is not available to the deployed agent.
Illustrative example: If only the third retrieval query strays, later failure should not assign equal responsibility to every correct evidence collection step.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Core mechanism
Why keep original rollout prefixes when a same-weight teacher sees a reflection patch?
SRPO: Self-Reflective Policy Optimization for Long-Horizon Reasoning
Show answer and example
Answer: The patch is hindsight training context for evaluating visited states. Rewriting student prefixes or giving the patch at deployment changes the task. Token teacher/old-student log ratios supply dense credit; reflection isn't independently proven truth.
Illustrative example: After a failed retrieval, a teacher may know the missed constraint while scoring, but the student didn't know it during the original choice.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Implementation details
Should SRPO rescore a teacher-generated ideal prefix?
SRPO: Self-Reflective Policy Optimization for Long-Horizon Reasoning
Show answer and example
Answer: No. Keep actual student on-policy prefixes; patch changes teacher context only. Detach reward difference and advantage. Local proxy axes do not generate real patches or execute a large tool model.
Illustrative example: Score the student's wrong third step as it occurred instead of replacing it with a correct one.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Method comparison
How does SRPO self-reflection differ from an independent external teacher?
SRPO: Self-Reflective Policy Optimization for Long-Horizon Reasoning
Show answer and example
Answer: Weights are shared and conditioning differs via feedback/patch, not additional parameters. A bad reflection can amplify shared bias; it isn't independent verification.
Illustrative example: Someone judging after seeing the outcome knows more, but isn't a second expert.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Basics
Why create a reflection patch after SRPO failure?
SRPO: Self-Reflective Policy Optimization for Long-Horizon Reasoning
Show answer and example
Answer: The patch converts coarse outcome feedback into privileged corrective context. The same model uses it to score actual student prefixes densely, not copy the failed trace.
Illustrative example: After a tool error, a parameter-check reminder can guide teacher rescoring.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
How does question-side ERPO regularization differ from constraining replies?
Beyond the Stability-Exploration Dilemma: Environmental Regularization for LLM Policy Optimization
Show answer and example
Answer: Answer KL limits output drift and exploration directly. ERPO weighs queries with reference likelihood and constrains induced question drift, changing update allocation across inputs. It isn't a renamed answer KL; typicality does not verify real query modeling.
Illustrative example: Reweight familiar questions that dominate training rather than constrain every answer to mimic the old model.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
Why is local ERPO typicality only a query-likelihood proxy?
Beyond the Stability-Exploration Dilemma: Environmental Regularization for LLM Policy Optimization
Show answer and example
Answer: Finite candidate features approximate input typicality to test flow and gradients, not natural-question LM likelihood. Use frozen reference weights, not weights inferred from test correctness.
Illustrative example: Typical feature frequency isn't the probability of a complete Chinese question.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
Can Query-KL and Policy-KL be interchanged in ERPO?
Beyond the Stability-Exploration Dilemma: Environmental Regularization for LLM Policy Optimization
Show answer and example
Answer: No. One constrains question distribution, the other conditional replies, with different gradient boundaries. Applying pressure to the answer scoring function isn't input-only control.
Illustrative example: Which question is sampled and how it is answered are different distributions.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
Why observe drift on the input side in ERPO?
Beyond the Stability-Exploration Dilemma: Environmental Regularization for LLM Policy Optimization
Show answer and example
Answer: Response regularization directly limits exploration. ERPO controls induced question-distribution shift with question likelihood weights and Query-KL, moving where regularization applies.
Illustrative example: Balance which questions receive learning pressure rather than forcing old answer wording.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
Why select patches on a development set and record EvoDAG after a same-batch check?
AutoSaddler: Automatic Harness Optimization with Durable Updates from Agent Execution Traces
Show answer and example
Answer: The original failures help diagnose direction but can be overfit by a targeted patch. Recompare on development data; the version graph records parent harness and evidence for future iteration and rollback. Bounded structured patches do not imply semantic safety, and final reporting tests cannot choose updates.
Illustrative example: After fixing three identical errors, check other development tasks for regressions and save the parent–child relation instead of overwriting the only harness file.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
What do local durable-update and reuse counts establish?
AutoSaddler: Automatic Harness Optimization with Durable Updates from Agent Execution Traces
Show answer and example
Answer: They show that the optimization loop, validation gate, and persistent reuse state run on a public mini-suite, without original GAIA2, SWE-Bench Pro, Terminal-Bench, or production harnesses. Diagnostic and write counts are not real LLM diagnosis, and proxy cost is not API cost, human efficiency, or online benefit.
Illustrative example: Twelve rule updates and 108 reuses verify the version path. Without real models and tools, the authors' nine-to-ten-point gain does not carry over.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
Which persistent state changes when editing harness code versus base-model weights?
AutoSaddler: Automatic Harness Optimization with Durable Updates from Agent Execution Traces
Show answer and example
Answer: Harness edits change how prompts, tool configuration, or middleware organize later execution. Weight updates change the model's behavior distribution. AutoSaddler iterates on the harness as a persistent code asset; workflow repair is not model training. The approaches can complement one another but require separate evidence, versions, and rollback targets.
Illustrative example: A recovery branch after failed tests changes workflow code. If the checkpoint is unchanged, its parameters have not learned error recovery.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why does AutoSaddler turn one task's failures into workflow improvements usable later?
AutoSaddler: Automatic Harness Optimization with Durable Updates from Agent Execution Traces
Show answer and example
Answer: Prompt, tool configuration, and middleware flaws can repeatedly cause long-task failures. Fixing one answer does not address shared causes. AutoSaddler proposes bounded harness edits from execution evidence and maintains persistent versions. Only execution validation and later reuse show a patch's effect; version counts are not capability.
Illustrative example: Repeatedly skipping pre-submission tests suggests a shared workflow change. Manually testing the current task fixes one deliverable, not future omissions.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
Why sample guidance depth by difficulty cluster rather than retain the same expert-prefix length for every task?
Agent-G²: Gaussian Guidance for Agentic Reinforcement Learning
Show answer and example
Answer: Too much prefix on easy tasks reduces independent exploration; too little on difficult tasks can produce only failures. Cluster success rates and variability from existing rollouts set the center and variance of a guidance band, then depths are sampled per task, avoiding extra per-item probes. Training prefixes are scaffolding and must not be provided in independent evaluation.
Illustrative example: An easy door task may need a very short hint; a complex multiroom task can need a longer start. Giving both half a prefix can overguide one and underguide the other.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
What does Agent-G² do if it samples a negative guidance depth or a proportion beyond the expert trajectory?
Agent-G²: Gaussian Guidance for Agentic Reinforcement Learning
Show answer and example
Answer: Bound the mean and clip Gaussian samples to [0,1], with a variance floor. Expert prefixes are training guidance; the hidden expert plan must not be supplied to policies in independent capability evaluation.
Illustrative example: Clip r=1.2 to 1 rather than indexing past the trajectory or treating overflow as extra oracle information.
Sections in the Chinese source: 核心公式, 复现边界. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
How does cluster-level guidance-band estimation differ in cost from repeated per-task probes?
Agent-G²: Gaussian Guidance for Agentic Reinforcement Learning
Show answer and example
Answer: Clusters reuse success rates and variation from existing policy rollouts, reducing extra per-task trials. Individual probes can be finer but cost additional rollouts. Neither fixed nor cluster-based help establishes optimal depth for each task.
Illustrative example: Similar tasks can share a depth distribution rather than repeatedly testing every task to find a starting point.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why does Agent-G² provide partial expert trajectories before exploration rather than relying only on random trial and error?
Agent-G²: Gaussian Guidance for Agentic Reinforcement Learning
Show answer and example
Answer: An expert prefix brings the policy to a state with more learning value, then the current policy explores from there. Existing rollouts estimate guidance proportions by difficulty cluster, balancing help with autonomy and avoiding costly per-example probes.
Illustrative example: A hard task can receive a longer start, while the policy takes over earlier on an easy task.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why does a large one-stage dynamic codebook still need collision codes and exposure-weighted updates?
Show answer and example
Answer: Fewer semantic steps reduce autoregressive work, but shared codes need extra disambiguation. Exposure-weighted updates adapt center resolution to real usage and change index semantics, so generation and serving mappings must stay synchronized.
Illustrative example: The main code identifies a popular group and a collision code identifies the item. Reading an obsolete mapping can return the wrong item despite faster decoding.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
Why define an explicit empty-bucket rule for weighted codebook updates?
Show answer and example
Answer: A weighted mean with zero total weight is undefined. Retain or handle empty centers under a defined rule rather than random fallback after division by zero. Publish EMA centers, assignments, and collision mappings in one version and disclose changes to the algorithm.
Illustrative example: Without any item entering code 7, there is no evidence for a new center; arbitrary reset invalidates existing paths.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
How do dynamic and frozen codebooks trade distribution adaptation against path stability?
Show answer and example
Answer: Frozen centers preserve meaning. Updates adapt to new content and exposure but may misalign older generators and indexes. Exposure weights can reinforce policy bias; measure update benefits, drift, and version maintenance separately.
Illustrative example: Holiday arrivals can justify moving centers, but old items changing code silently would make the generator's old probabilities wrong.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why is a one-stage large codebook shorter to decode but not equivalent to a unique item ID?
Show answer and example
Answer: One level shortens the path, but finite centers may each represent multiple items. Separate collision codes give uniqueness. A large vocabulary increases candidate and update costs, so fewer steps alone do not establish overall savings.
Illustrative example: A short book code can identify a bucket shared by two books; it cannot specify the display item without disambiguation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Core mechanism
How does RARE steer representations without rerouting MoE tokens?
RARE: Decoupling Representation Steering from Expert Routing in Mixture-of-Experts Language Models
Show answer and example
Answer: Direct edits can change router-visible coordinates. Project edits into the current router's nullspace, then remove newly visible drift at protected later layers. This separates behavior edits from route changes, not all nonlinear effects.
Illustrative example: Remove a style edit's routing component initially and correct drift after intervening transformations.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Implementation details
Why correct later protected layers after the first projection?
RARE: Decoupling Representation Steering from Expert Routing in Mixture-of-Experts Language Models
Show answer and example
Answer: Nonlinear propagation can create new router-visible components. At each protected layer, project the edited-minus-clean difference into that router's nullspace.
Illustrative example: The first router can stay unchanged while the next one needs correction.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Method comparison
How does RARE nullspace projection differ from reducing edit strength?
RARE: Decoupling Representation Steering from Expert Routing in Mixture-of-Experts Language Models
Show answer and example
Answer: A smaller edit can still shift logits. Nullspace projection gives RΔh=0 for the protected linear router. Behavioral improvement still needs measurement.
Illustrative example: A tiny nonzero score change can cross top-k; the nullspace removes that direction.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Basics
Why can direct activation steering change expert selection?
RARE: Decoupling Representation Steering from Expert Routing in Mixture-of-Experts Language Models
Show answer and example
Answer: The router reads hidden state. Editing it can move expert logits and alter the computation path, confounding behavioral effects.
Illustrative example: An intended style change may also send the token to different experts.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
Why bound action-level JSD weighting and preserve the original trajectory advantage's sign?
AUSO: Action-Level Unified Skill Optimization from Internalization to Utilization
Show answer and example
Answer: Distribution differences with and without skills show action sensitivity, not whether the action benefited. Preserve the outcome-advantage sign so local weights only adjust credit magnitude and a highly divergent step in a failed trace does not become positive reward. Separate internalization, exploration, and utilization to reduce persistent external skill-context costs.
Illustrative example: If an action in a failed run heavily depends on a skill, it can receive stronger corrective learning, not a success reward merely for depending on that skill.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
Why compare skill-conditioned and unconditioned policies at the same h_t and action-distribution granularity?
AUSO: Action-Level Unified Skill Optimization from Internalization to Utilization
Show answer and example
Answer: Changing other context confounds JSD. Compare the distributions at the same action boundary, and verify that actual w is nonnegative and bounded. Writing 1±β alone does not ensure the advantage sign is preserved.
Illustrative example: If one distribution sees a tool result and the other does not, the difference cannot be attributed only to skill S.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
Do AUSO's action-level JSD and whole-trajectory success tell us the same thing?
AUSO: Action-Level Unified Skill Optimization from Internalization to Utilization
Show answer and example
Answer: Success describes the outcome of the whole process. Action-level differences show a particular step's sensitivity to skill conditioning. A large JSD does not establish causal benefit; combine it with outcomes and bounded weighting rather than equating divergence with improvement.
Illustrative example: Only one tool choice in a successful trajectory may really be affected by the skill.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why does AUSO both internalize skills and retain an action-level skill utilization stage?
AUSO: Action-Level Unified Skill Optimization from Internalization to Utilization
Show answer and example
Answer: Retrieving external skills costs context, but fully internalized skills are harder to select per task. AUSO first uses skill-conditioned teachers for general internalization, then explores and compares action distributions with and without skills to adjust learning emphasis.
Illustrative example: Common steps can be learned into the policy while a complex action may still benefit from a particular skill hint.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
Why specify different causal boundaries when one autoregressive backbone outputs both policy and value?
SAPO: Single-Rollout Autoregressive Policy Optimization for Agentic Reinforcement Learning
Show answer and example
Answer: Action probabilities use only visible history before the decision, and value estimates must match the correct time. Post-action observations can leak future information into advantages. A shared backbone saves structure, but PPO, SARSA, and GAE still need state/action alignment; a single rollout does not remove bootstrap and termination semantics.
Illustrative example: The value before a command cannot already see the command's passing test result. Such credit would be precise only because the future leaked.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
Why retain trajectory boundaries when calculating GAE instead of concatenating episodes?
SAPO: Single-Rollout Autoregressive Policy Optimization for Agentic Reinforcement Learning
Show answer and example
Answer: Advantages accumulate later TD residuals. Concatenating episodes would propagate another task's rewards into the current one. Respect terminal boundaries for bootstrapping and accumulation. The deterministic local mini-suite is not a browser run or full RL training.
Illustrative example: The first record after task A ends belongs to B and is not A's future reward.
Sections in the Chinese source: 核心公式, 复现边界. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
How does single-rollout optimization differ from sampling many branches at one state for group comparisons?
SAPO: Single-Rollout Autoregressive Policy Optimization for Agentic Reinforcement Learning
Show answer and example
Answer: SAPO organizes policy, value, and temporal advantage from a single on-policy trajectory. Branch-group comparison needs multiple rollouts. Data requirements, value estimation, and credit assignment differ, regardless of whether final success rates match.
Illustrative example: An executed trajectory does not give the actual returns of untried alternatives at the same state.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why does SAPO use one autoregressive backbone for policy and value?
SAPO: Single-Rollout Autoregressive Policy Optimization for Agentic Reinforcement Learning
Show answer and example
Answer: A shared backbone reads action decisions and value signals at different causal positions and combines PPO, on-policy SARSA, and trajectory GAE. Sharing parameters does not erase the different input boundaries.
Illustrative example: Policy information available before action generation and later information permitted for value estimates must follow the protocol separately.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
What specialization does SAIM retain after OneModel shares a causal backbone?
OneModel: A Unified Foundation for Platform-Scale Multi-Scenario Ranking
Show answer and example
Answer: Scenario projections map heterogeneous events into one history space, but sharing every FFN activation can flatten differences. SAIM conditions shared FFN channels on scene embeddings. Global and recent states separately preserve stable and immediate signals.
Illustrative example: Illustration: Travel browsing and purchases can share history, while ads and content use different gates and recent flight booking remains in the local state.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
What should generate OneModel's SAIM gate, and why not use outcomes after prediction?
OneModel: A Unified Foundation for Platform-Scale Multi-Scenario Ranking
Show answer and example
Answer: Use the scenario embedding known at request time to modulate the shared FFN. Outcomes supervise only. Maintain causal masking and correct domain projections. Local genre-derived scenes are proxies, not real recommendation, ad, and merchant logs.
Illustrative example: Illustration: An ad-request flag is available now; selecting a gate because the user later purchased leaks future information.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
What time scales does OneModel global/local fusion preserve beyond a pooled user state?
OneModel: A Unified Foundation for Platform-Scale Multi-Scenario Ranking
Show answer and example
Answer: Global attentive pooling summarizes stable patterns, while recent states retain rapid changes. A single pooled vector may average away new needs. Genre-derived scenes lower density in the proxy; report the negative result without extrapolating it to real cross-business logs.
Illustrative example: Illustration: Long-term travel and a new renovation need can coexist without years of travel events dominating a single average.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why map heterogeneous scenario fields into uniform event tokens in OneModel?
OneModel: A Unified Foundation for Platform-Scale Multi-Scenario Ranking
Show answer and example
Answer: Different businesses log different fields and actions. Scene projections establish a common width and attach scene, action, time, and position before the causal backbone. Shared width does not equal shared meaning; gates and hierarchical states preserve differences.
Illustrative example: Illustration: A ten-second view and a purchase should not be two untyped IDs. Event metadata tells the model what each action means.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Core mechanism
Why mask conflicting teacher and progress rankings instead of trusting teacher?
Beyond Imitation: Filtering On-Policy Distillation by Reasoning Progress
Show answer and example
Answer: Style can receive teacher approval without useful progress. Independent ordering checks direction. Masking signals uncertainty, not proof a span is wrong, and the progress evaluator still needs validation.
Illustrative example: Don't reinforce a polished long explanation if it leaves the key constraint unsolved.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Implementation details
What identities must both R²-OPD rankings share?
Beyond Imitation: Filtering On-Policy Distillation by Reasoning Progress
Show answer and example
Answer: Score the same spans of the same trajectories and align by identity; define ties. Local candidates do not run a real progress judge.
Illustrative example: Match a span across rankings, not the third list position.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Method comparison
How does rank filtering differ from summing rewards?
Beyond Imitation: Filtering On-Policy Distillation by Reasoning Progress
Show answer and example
Answer: Filtering requires agreement to retain teacher signal. Addition lets a large score offset a poor one. Agreement doesn't prove either judge correct.
Illustrative example: A huge teacher score cannot buy back a span rejected by progress disagreement.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Basics
Why estimate reasoning progress independently in R²-OPD?
Beyond Imitation: Filtering On-Policy Distillation by Reasoning Progress
Show answer and example
Answer: Teacher preference needn't help solve the task. Compare independent progress and mask conflicting supervision rather than equate style with reasoning.
Illustrative example: A fluent explanation may score high with teacher while moving farther from the solution.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
Why use hinted versus unhinted regret for the environment designer rather than simply maximizing agent failure?
SPADE: Self-Play in Adaptive Synthetic Executable Environments
Show answer and example
Answer: Failure alone can reward impossible or overly hard environments. A hint that substantially improves a difficult unhinted outcome identifies a more learnable frontier. Environments must still execute, and privileged information stays in training comparisons rather than leaking into unhinted evaluation.
Illustrative example: A difficult maze solvable with direction hints can be useful. A maze with no exit fails both agents and should not score well just for being hard.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
May a SPADE agent retain hints in the no-hint evaluation condition?
SPADE: Self-Play in Adaptive Synthetic Executable Environments
Show answer and example
Answer: No. Isolate condition inputs. Privileged hints belong only to their training or comparison branch and must not enter the unhinted policy. Executable environment checks still matter; the local mini-suite did not run original external environments or large-model training.
Illustrative example: If an unhinted callback still receives the designer's saved plan, the return difference cannot be interpreted.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
Do maximizing failure rate and maximizing hint/no-hint regret favor the same environments?
SPADE: Self-Play in Adaptive Synthetic Executable Environments
Show answer and example
Answer: No. Failure alone can favor unsolvable or meaningless environments. Regret also requires privileged help to improve return, distinguishing solvable but currently difficult tasks. It still needs real execution and feedback rather than a fixed difficulty label.
Illustrative example: An environment that scores zero for both agents has a high failure rate but zero regret.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why does SPADE use the same LLM as environment designer and inference agent?
SPADE: Self-Play in Adaptive Synthetic Executable Environments
Show answer and example
Answer: The designer creates executable synthetic environments, the agent tries them, and the difference between hinted and unhinted rewards identifies frontier tasks. The aim is learnable challenges rather than arbitrary unsolvable problems.
Illustrative example: An environment that fails without hints but succeeds with them may be more informative than one that fails both conditions.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
Why do reverse-turn updates help keep decisions aligned with on-policy downstream continuations?
RTPO: Reverse-Turn Policy Optimization for Stabilizing Agentic RL Training
Show answer and example
Answer: Early action value depends on later policies. If downstream behavior has changed substantially, credit from old continuations can be inconsistent. A reverse tree handles turns near the outcome first, then propagates evidence backward so early decisions reflect the updated continuation. Update order does not remove importance or execution-alignment checks.
Illustrative example: After the last two steps learn recovery, assess opening decisions using that new recovery ability instead of values from the old failing continuation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
Why record both behavior policy and downstream continuation version for each RTPO turn?
RTPO: Reverse-Turn Policy Optimization for Stabilizing Agentic RL Training
Show answer and example
Answer: Early advantages based on mismatched later policies can lose on-policy correspondence. Reversing indices alone is insufficient: check tree nodes, state, and continuation ownership. Local control flow is not complete external-environment RL.
Illustrative example: After a node is reevaluated under a new continuation policy, its old continuation reward is not the current return for the same state.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
Are reverse-turn updates the same as training on reversed token text?
RTPO: Reverse-Turn Policy Optimization for Stabilizing Agentic RL Training
Show answer and example
Answer: No. Reverse order applies to decision updates and continuation dependencies. Causal language-model input still follows the legal trajectory. Reversing text changes the task and visibility and cannot substitute for reverse policy optimization.
Illustrative example: Updating turn three before turn two does not mean reading turn-three observations to predict a turn-one action.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why does RTPO organize multi-turn decisions into reverse updates?
RTPO: Reverse-Turn Policy Optimization for Stabilizing Agentic RL Training
Show answer and example
Answer: Early action value depends on downstream continuation. RTPO arranges rollouts in a sparse reverse tree and updates turns backward to keep decisions aligned with the downstream policy evaluating them.
Illustrative example: Handle the consequences of late actions first, then return to earlier decisions rather than treating the trajectory as unstructured text.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Core mechanism
Why fuse semantic and collaborative codes before conservative slate optimization in OGR?
Show answer and example
Answer: Semantic codes share content structure and collaborative codes capture behavior relations; encoding organizes the generative space. Planning treats a slate as one action to model item combinations, and conservative feedback updates limit drift from supervision. Representation, joint action, and stable learning are separate.
Illustrative example: Illustration: Four individually strong shoes can be redundant, while a slate can combine shoes, socks, and a pack without jumping to an unfamiliar combination for one reward.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Why must OGR's R(Y) and KL compare the whole slate rather than one item?
Show answer and example
Answer: Y is the slate, so reward, generation probability, and reference must apply to the same list or defined token decomposition with consistent IDs and deduplication. Mean item reward misses interactions and fixed score mixing is not policy updating. State the actual local path.
Illustrative example: Illustration: Five individually relevant duplicates can have high mean item score but poor slate value.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
Why does conservative SPA keep an SFT anchor rather than maximize reward freely?
Show answer and example
Answer: Sparse or misspecified rewards can favor extreme slates. KL limits drift from supervised behavior, trading improvement freedom for stability without fixing bad rewards. Local NDCG and Hit moved in different directions, not universal gains.
Illustrative example: Illustration: Novelty alone may favor irrelevant tail items; SFT preserves interest, but excessive constraint may block exploration.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why does OGR's TUSID encode semantics and collaboration instead of only quantizing content?
Show answer and example
Answer: Content generalizes to new items; collaborative codes preserve consumption relations. GL2P plans a slate and SPA adapts from feedback, distinct from coding. Richer codes can conflict and do not guarantee better pages.
Illustrative example: Illustration: Visually different items bought together can connect via collaboration; new items can borrow semantic structure.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why freeze shared multimodal features across Netflix MediaFM tasks rather than train a separate visual model for each?
Multimedia Asset Personalization via Multimodal Embeddings at Netflix
Show answer and example
Answer: A reusable feature layer amortizes media understanding. Downstream towers adapt users, queries, and assets, with query similarity complementing personalization in search. Sharing reduces duplicate encoding but cannot recover visual details absent from frozen features. Offline screening does not replace online validation.
Illustrative example: Illustration: Reuse a poster vector for home artwork and search; emphasize query match for 'seaside comedy' and long-term taste on home.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
Why calibrate all three vector scales when MediaFM combines two dot products?
Multimedia Asset Personalization via Multimodal Embeddings at Netflix
Show answer and example
Answer: User, query, and asset vectors need comparable scales or explicit calibration, otherwise one norm dominates the nominal α. Select checkpoints and fusion weights without tuning on test and bind frozen features to versions. Local movie-content proxies are not the production multimodal pipeline.
Illustrative example: Illustration: A query norm ten times larger can dominate a nominal 50/50 mix; normalization makes the coefficients interpretable.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
How does query similarity add current search intent beyond user-only personalization?
Multimedia Asset Personalization via Multimodal Embeddings at Netflix
Show answer and example
Answer: User similarity reflects long-term taste; query similarity adds the explicit current request. Balance them so neither erases the other. This does not establish a universal weight, and offline IPS and winner proxies do not replace online evidence.
Illustrative example: Illustration: A user who likes action searches for a warm comedy. Query match should override some action preference to keep search artwork relevant.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why does MediaFM provide shared media features rather than final recommendation probabilities?
Multimedia Asset Personalization via Multimodal Embeddings at Netflix
Show answer and example
Answer: Multimodal features describe content, while downstream towers adapt them to users, queries, and task objectives. Freezing avoids duplicate computation but limits missing details. Verify feature reuse separately from business outcomes.
Illustrative example: Illustration: The same poster vector supports home artwork and search, but high similarity does not make it optimal in every context.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
Why compare turn counts and response lengths only among successful traces rather than rewarding every short trace?
PlanPO: Group Planning-Aware Policy Optimization for Multi-Turn Agentic LLMs
Show answer and example
Answer: Short can mean efficient planning or premature abandonment. First separate successes by outcome, then compare turns and response length among comparable successes. Coarse-to-fine advantages supplement outcome, not correctness, and fewer tokens cannot excuse skipped evidence collection.
Illustrative example: Two successful bug fixes can differ by three wasted searches. A shorter trace that declares completion without editing files should get no efficiency reward.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
Why distinguish agent responses from environment observations when counting turns and response lengths?
PlanPO: Group Planning-Aware Policy Optimization for Multi-Turn Agentic LLMs
Show answer and example
Answer: The two axes measure interaction turns and per-turn generation. Counting long tool returns as agent output changes credit. Record success filtering, counting boundaries, and each advantage coefficient, not just total mean tokens.
Illustrative example: A tool can return a long page without the agent generating the same number of tokens.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
How does PlanPO efficiency among successes differ from rewarding the shortest answer?
PlanPO: Group Planning-Aware Policy Optimization for Multi-Turn Agentic LLMs
Show answer and example
Answer: A direct length reward can favor failure or early stopping. Comparing successful traces first requires completion, then optimizes turns and length. Length advantage is not global outcome advantage and must remain separate.
Illustrative example: An unexecuted one-line 'done' should not beat a successful multi-turn execution just because it is short.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why does PlanPO split planning efficiency into turn count and response length per turn?
PlanPO: Group Planning-Aware Policy Optimization for Multi-Turn Agentic LLMs
Show answer and example
Answer: Fewer environment interactions and shorter generated responses are distinct costs. PlanPO compares successes coarse-to-fine and combines efficiency with outcome advantage, retaining completion while improving the process.
Illustrative example: Two successful traces can differ in tool calls and explanation length; evaluate those dimensions separately.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Core mechanism
Why adjust weights based on saturation instead of always treating objectives equally?
Show answer and example
Answer: Mastered dimensions can waste budget needed elsewhere. Normalize per dimension, then reduce saturated goals by a specified statistic. High scores don't mean permanent mastery, and raw scale isn't mastery.
Illustrative example: Reduce formatting pressure if already solved, but check whether later reasoning training degrades it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Implementation details
Can SA-MRPO compute one saturation after aggregating rewards?
Show answer and example
Answer: No. That removes per-dimension budget allocation. Track each mean, variance and saturation before aggregation. Candidate formulas don't reproduce checkpoint training.
Illustrative example: One aggregate hides low correctness and high format performance.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Method comparison
How does saturation weighting differ from fixed reward weights?
Show answer and example
Answer: Fixed weights ignore current mastery; saturation weighting adapts by batch. It doesn't change ultimate importance or prevent forgetting.
Illustrative example: After downweighting a mastered skill, keep monitoring rather than declare it permanently solved.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Basics
Why track reward saturation in SA-MRPO?
Show answer and example
Answer: Already mastered objectives can crowd out unlearned dimensions. Standardize each dimension and weight by one minus saturation; the saturation estimate must be meaningful.
Illustrative example: If format nearly always passes, prioritize still-failing factual checks.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
Why score Evidence, Execution, and Invalidity separately instead of one process score?
TRCA: Transition-wise Rubric Credit Assignment for Long-horizon LLM Agents
Show answer and example
Answer: A transition can add evidence but fail to execute, or be well formed without advancing the task. Rubric components distinguish those cases and penalize invalid behavior. Breakthrough credit identifies key progress without a successful anchor, but still needs actual transitions, not fixed template scores.
Illustrative example: A search can reveal an error cause, followed by a command syntax failure. A total score alone obscures whether retrieval or invocation needs fixing.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
Why verify the sign and component logs when aggregating TRCA invalidity?
TRCA: Transition-wise Rubric Credit Assignment for Long-horizon LLM Agents
Show answer and example
Answer: Subtract r_invalid and add the other base and breakthrough terms as defined. Missing the negative sign rewards invalid behavior. Keep component scores linked to transitions, since a total can conceal mistakes. Fixed values are not real evaluation.
Illustrative example: With other terms fixed, increasing invalidity must lower total reward.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
How does TRCA's lack of a success anchor differ from feedback that needs a successful reference trace?
TRCA: Transition-wise Rubric Credit Assignment for Long-horizon LLM Agents
Show answer and example
Answer: Reference-based approaches first need a comparable successful execution. TRCA derives base and breakthrough rewards from actual transition rubrics without first finding a success. This is a feedback prerequisite, not proof of a higher task-success rate.
Illustrative example: Even when both full traces fail, a transition can collect useful evidence and receive rubric feedback.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why does TRCA define a rubric for every state transition rather than scoring only the outcome?
TRCA: Transition-wise Rubric Credit Assignment for Long-horizon LLM Agents
Show answer and example
Answer: Terminal outcomes hide important actions. Transition rubrics separately check evidence, execution, and invalid behavior and add breakthrough rewards. They do not require a successful anchor and supply more detailed credit.
Illustrative example: A step can gather key evidence while failing execution, with separate scores rather than zeroing both.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Core mechanism
Why does writing preconditions and postconditions first expose coverage gaps missed by direct test generation?
Grounding AI Agents in Contracts: An Empirical Evaluation of Spec-Driven Test Generation
Show answer and example
Answer: Specifications list input restrictions and guaranteed behavior per unit, then mark tested and untested items so tests target auditable coverage. Source interpretation can still be unfaithful. The paper's automated comparison did not include human edits, so don't treat optional developer steps as executed.
Illustrative example: A specification can flag empty input returning an empty list as an untested postcondition, whereas copying normal examples can miss it indefinitely.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Implementation details
Does compilation failure or modifying source count as bug detection?
Grounding AI Agents in Contracts: An Empirical Evaluation of Spec-Driven Test Generation
Show answer and example
Answer: No. The same test must compile and pass on fixed code and compile but fail a real assertion on buggy code. Illegal file edits invalidate runs; use an external trusted diff, not self-reports.
Illustrative example: An intentional call to a nonexistent API that breaks old compilation is not a valid regression test.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Method comparison
How does fixed-source input differ from exposing historical bugs to test generation?
Grounding AI Agents in Contracts: An Empirical Evaluation of Spec-Driven Test Generation
Show answer and example
Answer: The generator sees only repaired source and public requirements; the private evaluator uses buggy versions. Old patches or bug locations change difficulty and no longer test discovery under the same protocol.
Illustrative example: Knowing which line broke supports targeted tests, not independent contract inference.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Basics
What do Pre, Post, and Sug mean in specification-driven testing, and can suggestions replace behavior contracts?
Grounding AI Agents in Contracts: An Empirical Evaluation of Spec-Driven Test Generation
Show answer and example
Answer: Pre defines conditions before a call. Post specifies behavior given those conditions. Sug suggests verification. Suggestions are not contracts; define valid inputs and outcomes first, then mark tested and untested conditions to identify omissions.
Illustrative example: For sorting, Pre may require comparable elements, Post requires order and element preservation, and Sug proposes duplicate and empty-list tests.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why can separating planning, execution, and isolated reasoning reduce context pollution?
HyMem: Hierarchical Context Management for Long-Horizon Agents via Information Isolation
Show answer and example
Answer: The layers need different detail: planning needs goal progress, execution needs current actions, and transient reasoning need not persist. Structured summaries retain necessary state across refreshes without crowding long-term facts with guesses. Isolation does not guarantee correctness; summaries must preserve constraints and provenance.
Illustrative example: Example: Pass verified interface changes and open issues from debugging to planning, not every tentative hypothesis as a lasting fact.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
Do deterministic local PlanBench results show that GAIA or BrowseComp-plus ran?
HyMem: Hierarchical Context Management for Long-Horizon Agents via Information Isolation
Show answer and example
Answer: No. Local tests cover layered context, refresh summaries, and mechanism counts without browsers, external tools, or the paper's large model. Cost units and saturated scores belong to the controlled protocol, not real long-horizon token costs or paper Pass@1.
Illustrative example: Example: Counting a summary as one fixture event does not show that a real web evidence summary preserves the same facts at the same cost.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
What do structured progress summaries versus raw execution logs retain during handoff?
HyMem: Hierarchical Context Management for Long-Horizon Agents via Information Isolation
Show answer and example
Answer: Raw logs retain detail and many local attempts, leaving the next layer to identify what matters. Structured progress extracts completed work, current state, and open items, giving planning a stable starting point. Compression can omit constraints, so test whether the handoff supports continuation rather than counting text reduction.
Illustrative example: Example: Summarize installation logs as dependencies installed, tests not run due to missing credentials. 'Install succeeded' alone may be mistaken for completed verification.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Why can a multi-stage task leak irrelevant details into global decisions?
HyMem: Hierarchical Context Management for Long-Horizon Agents via Information Isolation
Show answer and example
Answer: Search, execution, and reasoning create many artifacts. Sending all of them to one context lets old logs or tentative assumptions distract planning. HyMem gives each stage the information it needs and uses progress records to continue, moving necessary evidence rather than sharing every token.
Illustrative example: Example: Hundreds of compiler lines may help the repair worker; global planning may need only the cause and affected module.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why does DrEM correct preference-flip risk instead of averaging prediction scores?
Show answer and example
Answer: Prediction noise can reverse pair preferences, and averaging can still learn the wrong order. DrEM selects perturbation-stable pairs, corrects the objective for flip risk, and applies consistency fusion to limit ranking drift. Correction assumes a noise estimate and becomes unstable near its boundary.
Illustrative example: A small CTR perturbation can reverse a close video pair; model that risk rather than trusting the observed order as a clean label.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Why separately check probability-to-logit conversion and `1−2ρ` in DrEM?
Show answer and example
Answer: Probabilities of zero or one cause infinite logits, while flip rate ρ near one half makes the inversion denominator singular. Document handling rather than silently producing huge gradients, and verify filtering, correction, and consistency fusion separately.
Illustrative example: A coin-flip preference contains no recoverable signal merely because it is divided by a tiny denominator.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
What assumptions does DrEM risk inversion add beyond perturbation augmentation?
Show answer and example
Answer: Augmentation exposes the model to noise. Inversion combines original and reversed-label losses according to estimated flip risk to recover the clean objective. It needs valid risk estimates and a nonsingular denominator; errors leave bias or amplify variance. Noise alone is not inversion.
Illustrative example: Correcting a true 40% flip rate as 20% remains biased. Random perturbations do not supply the inversion coefficient.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why study noise as pairwise preference flips in DrEM?
Show answer and example
Answer: Rankings depend on relative order; small errors can swap close candidates. Logit perturbation, preference-preserving filtering, and risk correction target this instability, while consistency fusion limits drift. Accurate individual predictions do not guarantee stable pairs, and flip estimates can be wrong.
Illustrative example: Scores only 0.01 apart can trade first place under small noise despite small individual errors.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Core mechanism
Why match full context and action before sharing TCAB feedback?
Fast A/B/n Testing: Exact Multi-Policy Comparison via Tree-Coupled Feedback Sharing
Show answer and example
Answer: Identical actions can yield different outcomes under different contexts. Tree maximal coupling matches complete pairs to share conditional rewards while preserving each policy's marginal trajectory. Coupled policies are deliberately correlated, and savings do not prove global optimality.
Illustrative example: Picking the same product for different users does not justify sharing one click observation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Implementation details
Why not count identical action strings as saved feedback calls?
Fast A/B/n Testing: Exact Multi-Policy Comparison via Tree-Coupled Feedback Sharing
Show answer and example
Answer: Reuse depends on full context and state conditions. The same action in different environments can yield different feedback. Compare actual shared queries and tree-edge differences.
Illustrative example: Two different users choosing A do not yield the same reusable observation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Method comparison
Does correlated coupling alter each policy's own trajectory distribution?
Fast A/B/n Testing: Exact Multi-Policy Comparison via Tree-Coupled Feedback Sharing
Show answer and example
Answer: Not necessarily. It changes joint dependence while preserving individual marginals. Equal final accuracy alone does not verify adaptive updates or marginal trajectory parity.
Illustrative example: Two draws can coincide more frequently while each retains its original probabilities.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Basics
Why share real feedback across adaptive policies?
Fast A/B/n Testing: Exact Multi-Policy Comparison via Tree-Coupled Feedback Sharing
Show answer and example
Answer: Separate A/B/n runs may pay repeatedly for identical context/action pairs. Coupling permits valid reuse while preserving each policy's marginal trajectory distribution.
Illustrative example: If two policies choose the same action in the same state, one feedback can update both.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Core mechanism
Why share a rubric subset within a rollout group?
Rubric Dropout: A Simple Way to Mitigate Reward Hacking in Rubric-as-Reward RL
Show answer and example
Answer: Relative rewards need common criteria; different random criteria can misrank peers. Cross-group randomization reduces fixed shortcuts, but doesn't guarantee valid remaining criteria or protect task performance.
Illustrative example: Omit brevity for both answers, not only one.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Implementation details
When should Rubric Dropout sample and save the mask?
Rubric Dropout: A Simple Way to Mitigate Reward Hacking in Rubric-as-Reward RL
Show answer and example
Answer: Sample one mask per rollout group, freeze for scoring and log seed. Never choose masks after seeing which response benefits. No local real rubric judge.
Illustrative example: Hiding difficult criteria only after observing failure is cherry-picking, not dropout.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Method comparison
How does group-shared dropout differ from independent masks per response?
Rubric Dropout: A Simple Way to Mitigate Reward Hacking in Rubric-as-Reward RL
Show answer and example
Answer: A shared subset keeps same-prompt responses comparable. Independent masks confound quality with scoring criteria. Randomize by group.
Illustrative example: Comparing A on format and B on correctness doesn't isolate answer quality.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Basics
Why randomly hide rubric criteria?
Rubric Dropout: A Simple Way to Mitigate Reward Hacking in Rubric-as-Reward RL
Show answer and example
Answer: Fixed proxies can be repeatedly exploited. Random subsets make a single shortcut unreliable. Hidden criteria are not unimportant; the visible reward composition changes during training.
Illustrative example: A canned phrase cannot reliably earn total reward if its easy criterion is often absent.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Core mechanism
Why simulate later dialogue before labeling a current PTO response?
Preference Tree Optimization: Enhancing Goal-Oriented Dialogue with Look-Ahead Simulations
Show answer and example
Answer: A pleasing immediate reply may harm long-term goals. Compare rolled-out user interactions and turn branch-level differences into current-response DPO preferences. Depth changes cost and horizon; simulator/oracle bias becomes training bias and is not real feedback.
Illustrative example: Guessing a need can read better in turn one, while simulating another turn may show clarification prevents a wrong direction.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Implementation details
Why preserve the mapping from current histories to future PTO branches?
Preference Tree Optimization: Enhancing Goal-Oriented Dialogue with Look-Ahead Simulations
Show answer and example
Answer: One preference pair compares actions from the same state. Continue each action in its own branch and do not leak future outcomes into policy inputs. Deterministic local evaluators replace the expensive oracle only to test tree and credit flow.
Illustrative example: If A prompts clarification and B confusion, attaching A's success to B reverses the preference.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Method comparison
What does adding more PTO branches expand compared with looking farther ahead?
Preference Tree Optimization: Enhancing Goal-Oriented Dialogue with Look-Ahead Simulations
Show answer and example
Answer: Width compares more actions at one state; depth observes consequences farther into the future. Both cost simulations and judgments but aren't substitutes. Shallow branches can miss delayed effects; one deep branch can miss better starts. This is a search distinction, not an empirical ranking.
Illustrative example: Five openings with one turn each increase width; one opening with many turns increases depth.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Basics
What do PTO's dialogue policy, virtual user and oracle do, and which is updated?
Preference Tree Optimization: Enhancing Goal-Oriented Dialogue with Look-Ahead Simulations
Show answer and example
Answer: The policy proposes replies, the virtual user continues possible interactions, and the oracle evaluates expanded branches. Their comparisons label preferences for the current state and update only the dialogue policy. Simulated users and evaluators provide training signals, not real user behavior or joint updates.
Illustrative example: A consultation agent asks two possible questions, the simulator answers each, and the oracle compares progress. Train the agent's current reply selection.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
Why project low-rank updates onto both singular complements rather than penalize overlap?
GCPO: Diagnosing and Constraining Subspace Geometry in Rollout RL for LLMs
Show answer and example
Answer: Soft cost permits principal-space drift at sufficient reward. Hard bilateral projection excludes it, keeping both input and output directions outside frozen principal spaces. Vector diagnostics are not layered LoRA training.
Illustrative example: Every valid update must avoid protected directions, rather than merely pay a penalty for touching them.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
Does local GCPO validate per-layer bilateral LoRA?
GCPO: Diagnosing and Constraining Subspace Geometry in Rollout RL for LLMs
Show answer and example
Answer: No. A vector policy restricts the leading right space of empirical update covariance as a proxy. The paper fixes left and right matrix projections; one-sided vector tests do not verify them.
Illustrative example: A parameter vector lacks two independent matrix axes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
How does hard orthogonality differ from a soft projection penalty?
GCPO: Diagnosing and Constraining Subspace Geometry in Rollout RL for LLMs
Show answer and example
Answer: Hard constraints exclude violating updates. Soft penalties allow an optimizer to trade violations against task gains. A larger penalty doesn't establish exact bilateral orthogonality.
Illustrative example: A soft update may briefly enter the principal space; hard projection removes that component every time.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
Why protect dominant pretrained singular spaces in GCPO?
GCPO: Diagnosing and Constraining Subspace Geometry in Rollout RL for LLMs
Show answer and example
Answer: Rollout update spikes in principal spaces may precede validation regressions. Restrict updates to bilateral orthogonal complements to avoid overwriting principal mappings. Correlation doesn't forbid learning in all dominant directions.
Illustrative example: Add task behavior in remaining directions while preserving important input/output mappings.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Core mechanism
Why compare full and degraded context margins rather than only raise chosen likelihood?
Show answer and example
Answer: Chosen likelihood may exploit language priors and ignore images. Context preference gain rewards the same pair changing preference with evidence while preserving the original ordering. It depends on appropriate degradation and does not itself establish correct visual understanding.
Illustrative example: If masking a chart leaves preference unchanged, language priors may dominate; correct evidence should provide extra support.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Implementation details
What must stay consistent across C²-DPO's margins?
Show answer and example
Answer: Use one prompt and one chosen/rejected pair under both contexts. Degraded context must not contain the answer. Local work does not complete fine-tuning the named multimodal model/data.
Illustrative example: Subtracting a full-image A/B comparison from degraded-image C/D does not isolate visual evidence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Method comparison
Is positive context preference gain the same as the chosen response winning under full context?
Show answer and example
Answer: No. CPG compares the two margins; winning needs a positive full-context margin. An improved but negative margin can still favor the wrong reply. Context gain and original preference ordering are separate constraints.
Illustrative example: A change from −3 to −1 gives positive gain without a chosen win.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Basics
Why does C²-DPO require a response pair and two contexts?
Show answer and example
Answer: Chosen/rejected compare responses to the same prompt; complete/degraded context varies evidence. They are independent axes. Hold the responses fixed to isolate context's effect.
Illustrative example: Compare red car and blue car with the original and degraded images, not with different questions.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Core mechanism
Why can GAS remove its entire auxiliary generation branch at deployment?
Show answer and example
Answer: During training it predicts continuous visual embeddings and backpropagates into a lower vision path shared with understanding. Its upper generation parameters are separate and not used by inference. Removing the branch retains the trained shared parameters.
Illustrative example: Train on image understanding and target visual embeddings, then deploy only image question answering with the learned shared vision path.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Implementation details
Why check inference dependencies before claiming GAS adds no deployment parameters?
Show answer and example
Answer: The generation upper layers, vision head, and EMA projector are removable only if understanding no longer reads them. Training still needs shifted prediction, EMA targets, and branch isolation. A local mirror-target diagnostic cannot validate the gains of large-scale generative data.
Illustrative example: Feeding an intermediate generation output back into understanding keeps that branch necessary at inference despite a claim of zero added parameters.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Method comparison
How does GAS next-embedding prediction differ from reconstructing the current patch?
Show answer and example
Answer: NEP predicts the next position from prior visual embeddings and allowed autoregressive context; an EMA projection supplies the target. Feeding the target at the same position changes what the model sees and can reduce the task to copying. Removability of the branch does not replace the shifted objective.
Illustrative example: To predict visual embedding i, provide earlier positions and allowed context, not embedding i itself.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Basics
Does GAS next-embedding prediction output discrete image tokens?
Show answer and example
Answer: No. It predicts continuous embeddings in the LM's visual input space, with cosine alignment to an EMA-projected target. It does not classify a discrete image codebook. Generation is auxiliary to understanding and deployment need not generate images.
Illustrative example: Predict the direction of a target patch embedding instead of choosing an index from thousands of image codes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Core mechanism
Why model reflection and backtracking as reversible memory-control actions?
LoongReflect: Boosting Long-Horizon Reflection in Search Agents via Global Perspective Distillation
Show answer and example
Answer: Long searches must choose external actions, which traces to retain, and which fork to return to. Making reflection and backtracking policy actions lets the agent learn when to change memory. A privileged teacher supplies training signals and outcome reward covers long-run success, but evaluation must not give the student the teacher's global view or future answers.
Illustrative example: Example: After a dead end, return to the previous branch and keep the reason for failure. Appending reflection text without changing trace state is not real backtracking.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Implementation details
Does a low mini-suite average cost establish savings for a real long-context model?
LoongReflect: Boosting Long-Horizon Reflection in Search Agents via Global Perspective Distillation
Show answer and example
Answer: No. The deterministic suite verifies reflection, backtracking, and state transitions without a real browser, external tools, or the paper's large model. Cost follows the diagnostic definition. When accuracy is saturated, inspect branches and traces; proxy counts do not translate directly into real tokens or API fees.
Illustrative example: Example: A rule branch billed a fraction of a cost unit does not mean real model reflection uses the same fraction of tokens or time.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Method comparison
How do the fast privileged-teacher channel and slow outcome-GRPO channel supervise different signals?
LoongReflect: Boosting Long-Horizon Reflection in Search Agents via Global Perspective Distillation
Show answer and example
Answer: The fast channel distills fine-grained reflection decisions from the teacher's view. The slow channel optimizes long-horizon policy using final trajectory outcomes. One provides local guidance, the other tests overall effectiveness. Privileged information belongs only with the training teacher or evaluator; the deployed student cannot read gold outcomes to choose backtracking.
Illustrative example: Example: A teacher can identify missing evidence at one search step. Final reward checks whether the complete answer succeeds. Online, the student sees only current observations.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Basics
Why does a long-horizon search agent need reflection that recovers its route, not just an explanation at the end?
LoongReflect: Boosting Long-Horizon Reflection in Search Agents via Global Perspective Distillation
Show answer and example
Answer: Multi-hop evidence gathering includes failed attempts. An early wrong turn can make later steps depend on bad evidence. LoongReflect changes current information and the continuation path so the agent can recover before the task ends. An accurate retrospective cannot improve the evidence gathering of an already completed run.
Illustrative example: Example: If search keeps finding reprints of the same article, return to the earlier branch and change sources before submitting, not after admitting the evidence is narrow.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why share user state between Sona generation and ranking, with teacher only in training?
Show answer and example
Answer: Separate models repeat history encoding and can learn inconsistent representations. One chronological compressed state supports joint generation and ranking, with training-only teacher supervision. It does not eliminate missed retrieval or compression loss; retain local negative results.
Illustrative example: Both paths respond to a shift from rock to classical without calling a large teacher online.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Does Sona call its teacher at serving?
Show answer and example
Answer: No. Teacher is training-only. Local work omits the full teacher, music tokenizer, and 15-plus-source production stack. Generation/ranking losses share state with distinct masks.
Illustrative example: Calling the large teacher online changes the cost contract.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
How does a unified model differ from deleting an intermediate ranker?
Show answer and example
Answer: Joint generation, ranking, and distillation must replace functionality and handle long history. Compare coverage and robustness; fewer models alone do not prove improvement.
Illustrative example: Dropping pre-ranking without joint training can just remove functionality.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Why use both SID generation and ranking in Sona?
Show answer and example
Answer: Share user state across candidate generation and fine ranking instead of duplicating representations. A large teacher guides training only; one model serves.
Illustrative example: Generate songs and score specific songs from the same user state.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
Why preserve model-specific sink scaling when using a sliding-window or causal mask?
Efficient Reinforcement Learning for Long-Horizon Tool-Use Agentic Tasks
Show answer and example
Answer: Sparse visibility and sink scaling affect different parts of attention. Mask changes do not justify dropping the model's scaling. Training still needs real environment state, long context, and GRPO data flow; a passing formula or kernel is not full on-policy rollout and tool-task verification.
Illustrative example: If the window retains recent text and sink but also removes special sink scaling, output changes combine two interventions and cannot be attributed to the window alone.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
Why not replace SinkFlex-RL token averaging with per-trajectory averaging and equal weights?
Efficient Reinforcement Learning for Long-Horizon Tool-Use Agentic Tasks
Show answer and example
Answer: The denominator is total valid length ΣT_i, giving long and short trajectories a different weighting from equal per-trajectory means. Preserve model sink scaling and verify the actual mask. The local mini-suite does not reproduce FlexAttention GPU memory curves.
Illustrative example: Trajectories of lengths 10 and 100 contribute differently to the token mean than to a mean with half the weight each.
Sections in the Chinese source: 核心公式, 复现边界. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
How do GRPO group-relative advantage and PPO with an independent value model differ in training dependencies?
Efficient Reinforcement Learning for Long-Horizon Tool-Use Agentic Tasks
Show answer and example
Answer: This approach builds relative advantages from groups without a separate value model, then applies a clipped policy objective and KL regularization. Changing advantage estimation alters rollout and memory needs; it is not an optimizer name change.
Illustrative example: Grouped returns provide a relative baseline, not a trained per-state critic.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why does SinkFlex-RL address both rollout environments and attention memory?
Efficient Reinforcement Learning for Long-Horizon Tool-Use Agentic Tasks
Show answer and example
Answer: Long tool traces require multi-turn state management and face context and training-memory bottlenecks. It combines environments, fresh trajectory groups, value-model-free GRPO, and sink-aware FlexAttention, not just a compressed mask.
Illustrative example: Saving attention memory still requires correct environment state for actual multi-turn policy updates.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Core mechanism
Why does MetaStrategy generate typed policies rather than final item order?
MetaStrategy: Generative Ranking with Executable LLM Strategies
Show answer and example
Answer: Typed objective weights, category preferences, and position constraints are explicit and compiled deterministically against real candidates. This bounds execution and makes failures auditable. The LLM interprets intent and the compiler executes it; evaluate them separately. Valid JSON proves interface compliance, not utility, and self-scores do not replace replay feedback.
Illustrative example: Illustration: For less repetition, emit a bounded category-diversity rule. If matching candidates are absent, expose the limit rather than inventing items.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Why must MetaStrategy's compiler reject unknown fields and out-of-range values?
MetaStrategy: Generative Ranking with Executable LLM Strategies
Show answer and example
Answer: Otherwise generated policies can escape the allowed space, breaking equivalence between evaluation and serving. Bind the schema and defaults to actual output and record failures rather than guessing intent. A JSON file does not implement online reward routing or teacher–student OPD.
Illustrative example: Illustration: Reject 'increase purchase weight without limit' and unknown fields rather than interpreting them as arbitrary executable expressions.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
What is auditable and reusable in MetaStrategy compared with direct item-list generation?
MetaStrategy: Generative Ranking with Executable LLM Strategies
Show answer and example
Answer: A list is bound to its candidates, whereas a policy can be inspected and reapplied after candidates change. Compiler expressiveness limits which preferences it can encode. The typed-bundle proxy is not full production distillation or equivalent intelligence.
Illustrative example: Illustration: 'At most one per author' works across candidate lists; a fixed ten-book list goes stale. The rule cannot interpret all content semantics.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why put objective weights, preferences, constraints, and position rules into typed JSON in MetaStrategy?
MetaStrategy: Generative Ranking with Executable LLM Strategies
Show answer and example
Answer: A fixed scorer is hard to steer on demand, and free text is difficult to execute reliably. Typed parameters separate semantic planning from deterministic ranking. Validity is only a format gate; still check ranges, replay, and feedback.
Illustrative example: Illustration: Raising a freshness weight with a placement limit is executable, unlike 'show more new items,' but it may still be bad for the user.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
What does IntHQ gain by using when, where, how, and via queries within multiple encoder layers rather than only four output heads?
Show answer and example
Answer: With heads only at the end, the encoder may compress every kind of evidence the same way. Task queries read separate behavior and context streams at multiple layers, so the task can guide evidence selection: time can favor periodicity and channel can favor entry points. Shared lower layers can capture common structure, but task conflict does not disappear automatically; verify that the queries really specialize.
Illustrative example: Illustration: In the same lunchtime search event, the time query cares about a lunch-break pattern and the channel query about search as the entry point. Letting both ask inside the encoder is more flexible than giving each a head on one summary.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why must IntHQ keep the task axis separate from the history axis in multilayer query aggregation?
Show answer and example
Answer: Each task query reads the relevant stream or layer's K/V and forms a task state. Softmax belongs on the candidate evidence axis. Normalizing over the four tasks as if they were historical records would make tasks compete. Role alignment matters more than matching final dimensions, and the public proxy does not cover all real spatiotemporal tasks.
Illustrative example: Illustration: The when query should distribute weight over historical times, not share a sum of one with the where query; otherwise adding a location task changes the time task's evidence budget.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
What structure does IntHQ's two-stream interaction preserve compared with concatenating behavior and context once?
Show answer and example
Answer: The streams distinguish events from context before task queries read across layers, avoiding undifferentiated mixing from the outset. One concatenation is simpler, but it is harder to track source and task conditions explicitly. Stronger structure does not guarantee results: the local public proxy regressed, and query names cannot replace evaluation.
Illustrative example: Illustration: A weekday lunch hour is context; recent restaurant clicks are behavior. The two streams can encode them separately before answering when and where, without forcing both to affect every target equally.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why are when, where, how, and via query roles rather than just labels in IntHQ?
Show answer and example
Answer: Queries extract different evidence from behavior and context streams, affecting intermediate representations instead of merely producing four final predictions. The two streams preserve distinct sources; multilayer interactions connect time, location, manner, and channel. The names suggest specialization, which still needs to be checked.
Illustrative example: Illustration: When someone returns depends more on intervals, whereas where they go depends more on location history. Two heads attached to the same fixed vector cannot separately guide the early read.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Core mechanism
Why might a likely buyer not be the best target under a limited outreach budget?
From Prediction to Incrementality: Causal Optimization for Large-Scale Targeting and Recommendation
Show answer and example
Answer: The policy cares about how much taking the action changes the outcome. High purchase probability may mean the user would buy without outreach. The method estimates treated and untreated outcomes and allocates the budget by uplift after an uncertainty penalty. A credible difference needs experimental support and identification conditions; two prediction heads alone do not make it causal.
Illustrative example: Illustration: A often buys either way, while B may buy only after seeing a relevant recommendation. B's raw purchase rate can be lower while B provides more incremental value for the same outreach slot.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Implementation details
Why should fixed-budget allocation check B but allow some budget to remain unused?
From Prediction to Incrementality: Causal Optimization for Large-Scale Targeting and Recommendation
Show answer and example
Answer: The constraint caps actions at B; it does not require all B. If risk-adjusted values are negative, do not send unhelpful messages just to fill the quota. Ranking and allocation must use consistent identities and cutoff features, without selecting people based on future test outcomes. The source's fixed-percentage proxy is not real long-term online incrementality.
Illustrative example: Illustration: If ten messages are available but only three users have positive risk-adjusted value, sending all ten can lower value. Unless the business explicitly requires it, the constraint does not force full use.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Method comparison
How does allocating by τ−λσ differ from ranking by mean uplift alone?
From Prediction to Incrementality: Causal Optimization for Large-Scale Targeting and Recommendation
Show answer and example
Answer: The uncertainty penalty deprioritizes poorly supported, possibly overestimated candidates and favors robust budget value. A mean-only policy may chase high-variance extremes. The penalty can also miss genuinely promising new groups and is not an exploration policy. A single proxy policy-value estimate cannot settle the trade-off universally.
Illustrative example: Illustration: Two users have the same estimated uplift, but one has abundant history and the other one response. Conservative allocation prefers the former; active exploration may require a separate budget.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Basics
Why does Incrementality define the value of outreach as μ1−μ0?
From Prediction to Incrementality: Causal Optimization for Large-Scale Targeting and Recommendation
Show answer and example
Answer: μ1 estimates the outcome under treatment and μ0 without it. Their difference distinguishes what the action creates from what would happen anyway. Give scarce capacity to people whose behavior can change, not merely those likely to respond. The two potential outcomes cannot be observed together, so estimation needs support and identification assumptions.
Illustrative example: Illustration: A would buy without a message; B changes their choice only with one. A high treated purchase rate does not imply high incremental value, so it is not enough to rank by that outcome alone.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why does Netflix GenRec not generate a recommendation list token by token despite using a decoder-only LLM?
GenRec: An LLM-Backed Recommendation Ranker at Netflix
Show answer and example
Answer: One prefill encodes text context, then a catalog-aware head scores items directly. This keeps semantic interactions while avoiding multi-step decoding and out-of-catalog generation. A joint language objective improves representation, not free-form serving; the local single seed does not establish Netflix-scale gains.
Illustrative example: Score every catalog movie once instead of writing one title and feeding it back to generate the next.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why isn't Netflix GenRec's local GRU comparison a parameter-matched architecture ablation?
GenRec: An LLM-Backed Recommendation Ranker at Netflix
Show answer and example
Answer: The experiment loads a larger public causal LLM and trains LoRA, while the baseline is ID-only GRU. Parameters and semantic inputs differ; it compares pipelines, not one isolated operator. Single seed, reduced catalog, and proxy rewards also preclude comparing directly to reported online percentages.
Illustrative example: LLM gains may come from title semantics or pretrained weights, not entirely from the catalog head.
Sections in the Chinese source: 本地复现, 对照口径. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does Netflix GenRec reward-weighted ranking differ from online RL over generated lists?
GenRec: An LLM-Backed Recommendation Ranker at Netflix
Show answer and example
Answer: Multiple rewards weight examples in supervised catalog ranking, alongside a language objective. There is no token-by-token rollout or list policy gradient; serving uses prefill scoring. The two reward uses have different probability and compute paths.
Illustrative example: Giving a satisfied example more loss weight is not generating ten online results and running PPO.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
How does Netflix GenRec's catalog-aware head prevent out-of-catalog hallucinations?
GenRec: An LLM-Backed Recommendation Ranker at Netflix
Show answer and example
Answer: A pooled prefill state scores only fixed catalog item IDs, not generated names. Text context and a language objective supply semantics. In-catalog outputs can still be irrelevant or unavailable; inventory and serving mappings need separate checks.
Illustrative example: An unreleased title known to the LLM cannot be recommended if there is no output column for it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why does DREAM translate bounded LLM semantic levels into parameters deterministically?
DREAM: A Dual-Loop Recommendation Evolution Framework Powered by Large Language Models
Show answer and example
Answer: Free-form text cannot reliably meet service schemas and ranges. Separate intent, planning, and compilation; the model proposes permitted directions and levels, while one outlet enforces whitelists and bounds. Only verified feedback enters policy memory. The LLM cannot arbitrarily rewrite production or amplify weights.
Illustrative example: An increase-diversity level compiles to a bounded diversification count instead of executing a vague instruction to promote novel content.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
How does DREAM compile semantic levels into legal, bounded parameters?
DREAM: A Dual-Loop Recommendation Evolution Framework Powered by Large Language Models
Show answer and example
Answer: Discrete levels must go through whitelists and deterministic translation. Ranking overrides use original weights and levels; category diversification count is `max(1, default_count + bc)`. Record executed parameters, not an unexecuted proposal, so feedback is attributable.
Illustrative example: A default count of one plus a level of −2 still compiles to at least one, never a negative serving parameter.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
Why is DREAM's shadow online gate not a real production A/B test?
DREAM: A Dual-Loop Recommendation Evolution Framework Powered by Large Language Models
Show answer and example
Answer: Independent shadow data reduce offline selection overfitting, but the public proxy and fixed reaction environment lack actual user feedback. Production double loops need real experiments. Local relevance improved while concentration rose and novelty fell; report all separately.
Illustrative example: Splitting validation users prevents repeated tuning on one group, but cannot measure adaptation, churn, or week-long live engagement.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why is DREAM a control plane over an existing recommendation pipeline?
DREAM: A Dual-Loop Recommendation Evolution Framework Powered by Large Language Models
Show answer and example
Answer: Intent and Meta Engine infer state and plans. Unified Outlet applies typed bounded parameters to existing retrieval, ranking, and reranking; underlying models still score items. Executability does not establish effectiveness without observed feedback.
Illustrative example: Control adjusts category weights and diversification for camping interest without replacing the model that scores every candidate.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why does ConnectionMind produce evidence paths and distill its path teacher into a GNN?
ConnectionMind: A General Social-Personalized Recommendation System with LLM Reasoning
Show answer and example
Answer: Paths expose intermediate graph relations and support step-level supervision and rewards. An LLM searches heterogeneous subgraphs for complex requests, while a distilled student handles ordinary traffic at lower cost. Paths must come from the visible graph; training shortest positive paths cannot be supplied as evaluation answers.
Illustrative example: A path through a followed creator and photography topic to a tutorial is inspectable. The student can reuse the relation pattern without full LLM search for every request.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
When can ConnectionMind's shortest-path demonstrations use target answers?
ConnectionMind: A General Social-Personalized Recommendation System with LLM Reasoning
Show answer and example
Answer: Training can derive step supervision from positive targets. Evaluation searches only the visible graph and current context, without held-out targets or gold paths. Keep format, recommendation, and path-shaping rewards distinct. Giving the agent a test answer path is diagnostic only.
Illustrative example: Showing how to navigate to a known item teaches training; revealing the correct movie node before test search cannot establish recommendation capability.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
How does ConnectionMind's stepwise policy compare with fixed GNN aggregation in scope and serving cost?
ConnectionMind: A General Social-Personalized Recommendation System with LLM Reasoning
Show answer and example
Answer: The policy selects task-dependent frontier expansions and relation paths; a GNN aggregates over fixed propagation structure. Full search is more flexible and expensive, so only complex minority traffic uses it. Locally, perfect path legality coexisted with negative recommendation results.
Illustrative example: Complex requests can explore friends and shared tags, while ordinary ones use a distilled model. Legal steps can still lead mostly to overly popular items.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why output graph evidence paths instead of only a social recommendation score in ConnectionMind?
ConnectionMind: A General Social-Personalized Recommendation System with LLM Reasoning
Show answer and example
Answer: A path shows which user, item, and semantic relations support the recommendation, enabling checks of edges, timestamps, and weights and supervision at each step. It is an explicit trajectory, not a fabricated explanation. Legality does not guarantee recommendation utility.
Illustrative example: Show the graph edges from a friend's photography collection to a tutorial. A claim that friends might like it is not equivalent evidence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Core mechanism
Why geometrically mix self-teacher and reference?
SR-OPSD: Self-Referenced On-Policy Self-Distillation
Show answer and example
Answer: Log-space combination treats tails and disagreement differently from arithmetic mixtures. Reference anchors, teacher guides and divergence regulates projection sensitivity. Equal endpoints do not make two interpolation methods equivalent.
Illustrative example: Distributions favoring different tokens can give very different intermediate geometric and arithmetic targets.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Implementation details
Why form SR-OPSD targets in log space and normalize explicitly?
SR-OPSD: Self-Referenced On-Policy Self-Distillation
Show answer and example
Answer: Power products underflow. Weighted log-space softmax yields a stable distribution on aligned teacher/reference vocabulary. No full model/Rényi experiments locally.
Illustrative example: Raw elementwise product without normalization isn't a probability distribution.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Method comparison
How does geometric interpolation differ from arithmetic averaging?
SR-OPSD: Self-Referenced On-Policy Self-Distillation
Show answer and example
Answer: Combine weighted log probabilities then normalize; arithmetic directly mixes mass. Low support from one side affects them differently, and formulas are not interchangeable.
Illustrative example: A nearly zero mass on one side can depress a geometric target more than arithmetic average.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Basics
Why keep a reference alongside SR-OPSD self-teacher?
SR-OPSD: Self-Referenced On-Policy Self-Distillation
Show answer and example
Answer: Self-teacher can drift or become overly strong. Reference anchors the geometrically interpolated target and Rényi sensitivity control instead of trusting self-teacher alone.
Illustrative example: Retain some prior support when teacher strongly favors a new mode.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Core mechanism
Why isn't matching parameter count sufficient for comparing long-context methods?
Cracks in the Foundation: Seemingly Minor Architectural Choices Impact Long Context Extension
Show answer and example
Answer: Normalization, GQA, pretraining windows, and sliding attention can all change long-context behavior. A controlled model pool isolates them; one model's success does not generalize a positional or attention modification.
Illustrative example: A longer-pretrained model beating a shorter-trained model does not isolate the extension method.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Implementation details
Does a passing local mini-suite establish the 7B pool's conclusions?
Cracks in the Foundation: Seemingly Minor Architectural Choices Impact Long Context Extension
Show answer and example
Answer: No. It verifies core data flow, not paper-scale checkpoints or full benchmarks. Causal attribution needs controlled training, matched extension protocols, and task evaluation.
Illustrative example: Accuracy on 64 diagnostic examples is not a controlled 7B long-context experiment.
Sections in the Chinese source: 本地复现, 复现边界. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Method comparison
How does changing normalization within the pool differ from comparing released models?
Cracks in the Foundation: Seemingly Minor Architectural Choices Impact Long Context Extension
Show answer and example
Answer: Released models can also differ in data, scale, and schedule. Controlled changes improve attribution, and interactions mean single-factor effects need not add.
Illustrative example: Swap normalization alone, then test whether the direction persists with GQA.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Basics
Why use a controlled OLMpool model pool?
Cracks in the Foundation: Seemingly Minor Architectural Choices Impact Long Context Extension
Show answer and example
Answer: Real models often differ in normalization, GQA, training length, and sliding windows simultaneously. The pool isolates these choices and their combinations for long-context extension.
Illustrative example: If architecture and training length both change, do not attribute the result to one toggle.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Core mechanism
Why are release monitoring and rollback needed in addition to Champion–Challenger evaluation?
Show answer and example
Answer: An offline candidate that beats the champion may still be unstable after deployment or over long interactions. Monitoring must allow traceable restoration of the old version when regressions appear. Version observations, planning, memory, verification, and recovery assets rather than only model names. Release governance is part of the evolution loop, not replaced by one passing candidate test.
Illustrative example: A challenger saves calls during development but loops on permission errors in production. Monitoring should identify its version and roll back instead of allowing unbounded trial and error.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Implementation details
Do local version and rollback states demonstrate real long-term online self-evolution?
Show answer and example
Answer: No. Deterministic local PlanBench verifies mechanism branches, version state, and reporting, with no real browser, service, or YC-Bench. Proxy costs, release flags, and rollback counts are control evidence, not online risk or success metrics. Do not present the paper's online or offline results as local measurements.
Illustrative example: Rules can change a state from challenger to champion. Without running actual models and tools, this does not establish deployment of a stronger agent.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Method comparison
How does Champion–Challenger differ from overwriting the current policy each round?
Show answer and example
Answer: Keep the champion as a verified baseline, compare challengers before release, and roll back under runtime monitoring if needed. This reduces persistent contamination by unverified changes. A release gate needs actual evaluation and version evidence rather than favorable self-assessment; online monitoring does not authorize arbitrary external deployment.
Illustrative example: If a recovery policy validates well but regresses after release, you should be able to restore a specific old version rather than having only an overwritten configuration.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Basics
Why is a Loop Policy more than a system prompt?
Show answer and example
Answer: Observation, planning, memory, action, verification, and recovery assets jointly shape long-horizon behavior. Version both components and combinations. Prompt changes may miss the real bottleneck; executable versions, comparison, and monitoring make evolution traceable. Complete files do not imply effective behavior.
Illustrative example: A prompt may say 'retry after failure,' but if recovery middleware cannot identify timeouts, the actual loop may still not recover.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Core mechanism
Why reward predictive KL rather than token deletions in DistillCache?
DistillCache: KL-Guided Adaptive KV-Cache Eviction for Memory-Efficient LLM Inference
Show answer and example
Answer: Deletion is a resource action; retaining predictive information is the goal. Compare full and evicted distributions per step so the policy can trade behavior loss against budget. The reference is training-only, and low KL is not universally lossless.
Illustrative example: Padding and a key ID both save one KV, but only the latter may sharply change next-token probabilities.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Implementation details
How sanity-check DistillCache's negative KL?
DistillCache: KL-Guided Adaptive KV-Cache Eviction for Memory-Efficient LLM Inference
Show answer and example
Answer: Equal distributions yield zero. Larger KL gives a lower reward. Use −KL(p_full||p_evict), not the reverse or a positive sign. Mini-suite does not prove policy training or deployment gains.
Illustrative example: Moving evicted predictions farther from full should not raise reward.
Sections in the Chinese source: 核心公式, 复现边界. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Method comparison
Would KL-guided and minimum-cache rewards prefer the same policy?
DistillCache: KL-Guided Adaptive KV-Cache Eviction for Memory-Efficient LLM Inference
Show answer and example
Answer: Not generally. One penalizes prediction drift; the other rewards space. Compare at fixed budget or explicit quality limits.
Illustrative example: Deleting all KV minimizes cache but can strongly change predictions.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Basics
Why treat KV eviction as a multistep decision?
DistillCache: KL-Guided Adaptive KV-Cache Eviction for Memory-Efficient LLM Inference
Show answer and example
Answer: A deletion changes later available history. Local salience may not predict future loss. Stepwise full-versus-evicted KL trains a policy for later prediction.
Illustrative example: A currently unused KV can become answer evidence and may not be recoverable after deletion.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why generate a servable SID before optionally generating explanation copy in PushDualGen?
Show answer and example
Answer: SID identifies the recommendation and lets it enter serving immediately. A gate adds text only when needed, conditioned on that SID and history, saving decoding elsewhere. Fusion and consistency keep copy about the selected item; the recommendation remains usable without text.
Illustrative example: Illustration: A feed can return the item code alone, while a notification generates a reason about those running shoes, not an unrelated novel.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Why define copy tensors and losses explicitly when PushDualGen has g=0?
Show answer and example
Answer: The skipped branch cannot require missing text; define masks and fallback states. At g=1, generate from the same SID and history. Placeholders are not explanations, and the local core scorer does not prove complete online copy generation.
Illustrative example: Illustration: A zero vector can mark missing text, not justify decoding generic praise and claiming personalized consistency.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
What budget and coverage trade-offs come from on-demand copy versus always generating explanations?
Show answer and example
Answer: Gating saves average decoding cost by reserving copy for useful scenarios, but some users may miss helpful explanations. Validate gating, caching, and scores for both modes. Do not present latency from one and effectiveness from the other as one path.
Illustrative example: Illustration: Report item-only feed speed separately from notification engagement with a generated sentence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Does a valid PushDualGen SID guarantee correct copy?
Show answer and example
Answer: No. Catalog legality identifies a servable item. Copy also needs factual consistency, user evidence, and no exaggeration. Check identity and explanation quality independently.
Illustrative example: Illustration: Ordinary running shoes are a valid item, but claiming they are fireproof fails factual verification.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why merge as sum/√n and distill a full-token teacher?
Show answer and example
Answer: Grouping bounds tokens, and square-root normalization moderates sum growth while retaining group-size effects. Attention and ranking distillation approximate useful fine relations without reversing compression.
Illustrative example: Merge four browses into one; teacher sees internal relationships the student cannot recover exactly.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
How align attention distillation across different lengths?
Show answer and example
Answer: Map teacher positions into student groups; do not subtract mismatched matrices. Preserve masks and sizes. Local work lacks production long histories and traffic.
Illustrative example: Specify which four teacher entries correspond to one merged slot.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does sum/√n differ from average pooling?
Show answer and example
Answer: Mean divides by n; square-root normalization retains different energy scaling as group size changes, depending on correlation.
Illustrative example: Repeating a vector changes the two resulting norms differently.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why let TM20K's teacher keep all tokens while the student merges?
Show answer and example
Answer: Full-token attention and ranking provide detail for distillation. Student compression controls serving budget but does not preserve every event.
Illustrative example: Teacher reads 20,000 events; student learns from grouped summaries.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Core mechanism
How can Autonomy-of-Heads assign sparse patterns without calibration examples?
Autonomy-of-Heads: Data-Free Sparse Attention from Frozen Query-Key Geometry
Show answer and example
Answer: It analyzes frozen Q/K projections and uses spectral effective rank to characterize potential matching geometry, separating retrieval-oriented from streaming heads. No sample-based calibration or runtime selector is needed, but real tasks must validate the pattern.
Illustrative example: Two heads with equal budgets can emphasize very different numbers of directions; assign patterns accordingly and test evidence retention.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Implementation details
Can singular values be used as probabilities directly for spectral entropy?
Autonomy-of-Heads: Data-Free Sparse Attention from Frozen Query-Key Geometry
Show answer and example
Answer: Normalize them into a distribution, or rescaling the matrix changes entropy's meaning. Handle degenerate cases. Local mini-suite checks data flow, not checkpoint quality.
Illustrative example: [2,2] and [20,20] both normalize to [0.5,0.5].
Sections in the Chinese source: 核心公式, 复现边界. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Method comparison
How does weight-based head assignment differ from dynamic attention selection?
Autonomy-of-Heads: Data-Free Sparse Attention from Frozen Query-Key Geometry
Show answer and example
Answer: Weight geometry gives stable assignments and avoids per-input selector work. Dynamic methods adapt to content but pay selection cost. Neither accuracy follows from calibration cost alone.
Illustrative example: A fixed retrieval head always reads far away, while a dynamic gate may allocate different ranges for short and long prompts.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Basics
What does data-free omit in Autonomy-of-Heads?
Autonomy-of-Heads: Data-Free Sparse Attention from Frozen Query-Key Geometry
Show answer and example
Answer: It uses frozen Q/K geometry rather than calibration examples and runtime gating to choose sparsity. This does not remove model pretraining or post-sparsification evaluation.
Illustrative example: Assign head types from weights before deployment, then check actual answers.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
Why keep hidden references on the environment and judge side in simulations with asymmetric information?
ResidencyRL: Reinforcement Learning in Simulated Clinical Environments
Show answer and example
Answer: The policy must gather evidence through questions, tools, and observations. Direct reference access bypasses long-horizon decision making. Isolate scoring dimensions before aggregation to reduce contamination, but the reward formula is not the full environment or professional capability. Local diagnostics are not clinical efficacy or decision reliability.
Illustrative example: If a state should be revealed only after querying it, do not provide the reference summary in the policy prompt and call the result autonomous evidence collection.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
Why cap the ResidencyRL penalty with min(3,P) rather than arbitrarily clipping total reward?
ResidencyRL: Reinforcement Learning in Simulated Clinical Environments
Show answer and example
Answer: First calculate three times the weighted normalized dimension scores, then subtract a penalty capped at three. Preserve order and the distinct terms. Hidden scenario information is for the environment and judges; isolated judge calls must not pass the reference to the policy.
Illustrative example: Increasing P from 4 to 10 still deducts 3; it does not change the other dimensions' scores.
Sections in the Chinese source: 核心公式, 背景与主要改动. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
How does aggregating six reward dimensions differ from implementing the full simulation training algorithm?
ResidencyRL: Reinforcement Learning in Simulated Clinical Environments
Show answer and example
Answer: Aggregation combines already obtained normalized scores and penalties. The method also requires scenarios, actual interactions, separate judges, and policy updates. Fixed scores or a loss wrapper cannot replace that chain.
Illustrative example: Combining six numbers into R does not establish that the model performed and learned from a simulated interaction.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why does ResidencyRL use multi-turn simulation instead of only static knowledge questions?
ResidencyRL: Reinforcement Learning in Simulated Clinical Environments
Show answer and example
Answer: Active inquiry, tool use, and long-horizon decisions need observed interaction effects. The method connects scenario generation, information-asymmetric environments, isolated multidimensional judges, and GRPO to train an interactive policy rather than memorizing answers.
Illustrative example: The agent must decide what to ask next from new observations rather than receiving all hidden scenario information up front.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Core mechanism
Why condition episodic retrieval on the current phase instead of querying the original issue every step?
Coupling Planning with Episodic Memory in LLM Agents for Software Issue Resolution
Show answer and example
Answer: Localization, editing, and verification need different evidence. Static retrieval can repeat topically related but phase-inappropriate traces. Hierarchical planning constrains retrieval by phase, detects stalls from behavior, and replans. Real execution verdicts decide whether repair succeeded, not phase changes.
Illustrative example: Example: Localization needs files and verification needs regression tests. Repeatedly searching issue text can keep returning localization experiences.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Implementation details
Can perfect local phase/retrieval scores be compared with solving 25 more SWE-bench issues?
Coupling Planning with Episodic Memory in LLM Agents for Software Issue Resolution
Show answer and example
Answer: No. Deterministic PlanBench tests phase-conditioned retrieval, stalls, and replanning without repository containers or paper models. Proxy cost is not tool tokens. Report traces and branch evidence rather than call saturation issue-resolution capability.
Illustrative example: Example: Selecting a phase template and completing a three-step fixture does not establish locating, editing, and testing an unknown repository.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Method comparison
What is the trade-off between trace-based stall detection and rewriting the whole plan after any failure?
Coupling Planning with Episodic Memory in LLM Agents for Software Issue Resolution
Show answer and example
Answer: Trace statistics recognize repeated nonprogress and allow targeted replanning. One failure can be useful evidence rather than a bad direction. Full rewrites repeat work, while never changing can stall indefinitely. Use actual verdicts rather than self-reported recovery.
Illustrative example: Example: The same test error after two identical patches suggests another approach. A new first-time error can be normal progress.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Basics
What reusable experience does PMCoder retain across locating, fixing, and testing an issue?
Coupling Planning with Episodic Memory in LLM Agents for Software Issue Resolution
Show answer and example
Answer: A long fix yields clues, failed attempts, and test conclusions. Restarting from the issue repeats searches and mistakes. Phase-organized plans and reads give each step the right history. Memory supports execution but current code and tests still decide if the new issue is fixed.
Illustrative example: Example: Remember the parser location and failing boundary input rather than search the whole repo again.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why distill both current rollouts and actual exposure candidates in Gryphon-v2?
Show answer and example
Answer: A changing generator proposes items rare in old logs; exposure-only training may not score its outputs well. Rollout-only training discards established coverage. One teacher calibrates both distributions, and a shared encoder links generation and ranking. Teacher bias can still transfer.
Illustrative example: Rollout supervision covers niche movies proposed by the new generator, while real exposures preserve knowledge of existing popular ones.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Why keep Gryphon-v2 rollout and impression distillation provenance separate?
Show answer and example
Answer: Counts, weights, and candidate identities from different distributions affect the loss. Compare teacher and student on matching items and cutoffs and track which source dominates. Both objectives updated locally, but quality regressed; updates are not gains.
Illustrative example: Sixty thousand distilled rollout items are not 60,000 real impressions and must not be labeled user clicks.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
Why does dual-source distillation better reflect generator serving candidates than logged impressions alone?
Show answer and example
Answer: Old impressions are policy-selected and can miss the current rollout distribution. Combined training sees actual exposure evidence plus the generator's outputs. The teacher may be unreliable in new regions; the local comparison was negative despite broader coverage.
Illustrative example: Niche instrumental candidates missing from old logs get rollout supervision, but can inherit teacher bias.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Why do Gryphon-v2's generation and ranking heads share a history encoder?
Show answer and example
Answer: Re-encoding the same history wastes computation and can produce inconsistent user states. A shared encoder supports SID nomination and ranking, but the ranker still depends on the generated candidate set and cannot recover omissions.
Illustrative example: Both heads see a user's recent switch to jazz, but missing jazz items cannot be reranked.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Core mechanism
How avoid treating shortest consensus as truth without gold?
Show answer and example
Answer: Consensus selects a reference and repair targets as pseudo-supervision, not correctness verification. Shared errors may be reinforced. Report internal agreement separately from external holdout and keep gold out of student inputs.
Illustrative example: Several responses can use the same wrong formula; the shortest still isn't a standard solution.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Implementation details
Can U-OPSD use gold before choosing an internal reference?
Show answer and example
Answer: No. Use predictions only for consensus, thresholds and length. Gold is for independent evaluation. Deterministic mini-suite isn't full multi-turn self-distillation.
Illustrative example: Filtering correctness before voting smuggles answer information into training.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Method comparison
Why distinguish shortest consensus answer from arbitrary shortest answer?
Show answer and example
Answer: First require majority agreement, then minimize length within eligible outputs. The shortest unrestricted may be an empty or outlier answer. Consensus isn't gold.
Illustrative example: A short erroneous refusal doesn't become privileged guidance because it saves tokens.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Basics
Why is U-OPSD privileged view still externally unsupervised?
Show answer and example
Answer: It comes from internal sampling consensus, without external answers/reward or a larger teacher. One view sees an extra internally produced signal, not independently verified truth.
Illustrative example: A short self-selected consensus solution can still be a shared error.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Core mechanism
Why combine group and anchor ranking rather than ask for absolute scores?
Show answer and example
Answer: Comparative judgments can be more stable than absolute scale. Group competition gives relative position; anchors give cross-group reference before centering. This still depends on candidates and judge, not absolute truth or full paper evaluation.
Illustrative example: Two group winners may have very different quality relative to a shared anchor.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Implementation details
Why check group size and rank direction before converting RRC ranks to rewards?
Show answer and example
Answer: K−1 requires more than one sample. Declare whether rank one is best. No local real judging or complete capability integration.
Illustrative example: A reversed formula can give the best rank the lowest reward.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Method comparison
What do group self-competition and anchor ranks each provide?
Show answer and example
Answer: Group comparisons separate current candidates; anchors bridge groups where ranks alone hide absolute level. Specify weights; rank gaps aren't equal quality distances.
Illustrative example: An all-bad group still has a winner; an anchor can show it remains poor.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Basics
Why derive RRC generative reward from relative ranking?
Show answer and example
Answer: Generative judges may compare reliably without calibrated standalone scores. Within-group ranks plus anchors yield centered rewards while exploiting comparative strength.
Illustrative example: A judge may prefer one summary reliably but inconsistently use a ten-point scale.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Core mechanism
Why compare matched views with and without English solutions?
Show answer and example
Answer: Reference-induced distribution shifts can identify reasoning transitions rather than superficial translation. Concentrate privileged distillation there and keep reference anchoring. Match all other inputs and keep solutions teacher-only.
Illustrative example: Translated connectives and words determining operation order may shift differently after adding a solution.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Implementation details
What must the two RP-OPSD teacher views share?
Show answer and example
Answer: Same model, question and student prefix; remove only reference conditioning. Student rollout cannot see reference answers. Local deterministic proxies aren't real multilingual teachers/corpora.
Illustrative example: Changing both model and question prevents attributing probability changes to the English reference.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Method comparison
How does pivot detection differ from uniformly imitating all English tokens?
Show answer and example
Answer: Uniform imitation can copy style and translation artifacts. Conditional shifts highlight important positions while retaining reference anchoring. Shifts aren't causal truth, so validate teacher quality.
Illustrative example: Different English wording with unchanged mathematical state need not rewrite every student word.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Basics
Why distill cross-lingually around reasoning pivots in RP-OPSD?
Show answer and example
Answer: Surface wording differs across languages; state-changing tokens matter more. Compare matched teacher distributions with/without the English reference to locate large information-induced shifts.
Illustrative example: A connective and a key equation need not receive equal credit based on text differences.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Core mechanism
Why copy the delta rather than every teacher preference?
On-Policy Delta Distillation for Multilingual Math Reasoning
Show answer and example
Answer: Comparing before/after logp isolates directions associated with post-training instead of inherited language/style/knowledge. It needs comparable checkpoints and isn't proven pure capability. Local small distributions don't verify language transfer.
Illustrative example: If RL improves conclusions but politeness is unchanged, the delta avoids treating the greeting as a new skill.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Implementation details
May OPD² use arbitrary teacher families for its two probabilities?
On-Policy Delta Distillation for Multilingual Math Reasoning
Show answer and example
Answer: Not silently. Use paired base and post-trained checkpoint on the same prefix with aligned vocabulary/conditioning. Local candidates do not load multilingual math checkpoints.
Illustrative example: An unrelated architecture as base confounds family calibration with the training delta.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Method comparison
How does delta distillation differ from copying the post-trained distribution?
On-Policy Delta Distillation for Multilingual Math Reasoning
Show answer and example
Answer: It subtracts the teacher base; direct matching includes old knowledge and preferences. Subtraction may discard useful information or retain training noise, so not universally better.
Illustrative example: Learning the update is not copying the whole teacher knowledge base.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Basics
Why distill the teacher's post-training change from base in OPD²?
On-Policy Delta Distillation for Multilingual Math Reasoning
Show answer and example
Answer: Teacher's preexisting preferences may not come from target training. The delta emphasizes what changed instead of transferring all language/style bias; matching checkpoints is essential.
Illustrative example: An unchanged long-sentence preference should not count as newly acquired skill.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Core mechanism
Why control DASH horizon by local disagreement rather than propagate everything?
Show answer and example
Answer: Later teacher feedback may help or become unreliable after divergence. Detached gating varies effective horizon without extra teacher calls. Fixed long or short windows can mix irrelevant signals or miss important outcomes. Gate is not truth.
Illustrative example: Carry future credit across consistent steps but weaken propagation at a severe divergence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Implementation details
How should DASH handle boundaries and gate gradients?
Show answer and example
Answer: Recur backward with no next term at sequence end, detach gate and block padding or cross-response propagation. Candidate positions aren't long-token traces.
Illustrative example: Do not carry future credit from the next answer into the last token of this one.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Method comparison
How does DASH's continuous gate differ from fixed discount?
Show answer and example
Answer: Fixed discount propagates equally by position; gate uses relative local disagreement to vary horizon. Proxy credit is not always correct and doesn't add teacher rollouts.
Illustrative example: Consequences at equal distance can propagate differently under different local disagreement.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Basics
Why propagate later distillation differences to earlier positions in DASH?
Show answer and example
Answer: Early choices have multi-step consequences. Local disagreement gates a backward recurrence with adaptive horizon, without extra teacher passes.
Illustrative example: An early fork causing repeated later errors should see some of their credit.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Core mechanism
How does MACRO change input computation without changing weights?
MACRO: Markov Chain Routing of Transformer Layers
Show answer and example
Answer: A context-conditioned routing program can skip, repeat, or residual-add layers. Feedback updates route probabilities and search selects programs; the backbone is fixed. Use validation, not isolated test gold, for selection.
Illustrative example: Two prompts can execute different paths through identical layers.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Implementation details
Can MACRO greedily concatenate the highest-probability operation at each step?
MACRO: Markov Chain Routing of Transformer Layers
Show answer and example
Answer: Not generally. Top-k Viterbi preserves Markov dependencies. Locally greedy choices can miss a better full program; the classification mini-suite is not the paper's scale.
Illustrative example: A likely repeat now can reduce the score of all later reachable paths.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Method comparison
What does routing search change compared with fine-tuning?
MACRO: Markov Chain Routing of Transformer Layers
Show answer and example
Answer: It changes how fixed layers execute, not internal parameters. Costs and generalization can differ, and a route is not a new trained backbone.
Illustrative example: Rearranging blocks differs from reshaping them.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Basics
Why treat layer execution as a program?
MACRO: Markov Chain Routing of Transformer Layers
Show answer and example
Answer: Context-conditioned Markov routes support skip, repeat, and residual-add rather than one fixed order. Feedback updates routes and candidate decoding chooses programs.
Illustrative example: One input can skip a layer and repeat another.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Core mechanism
Why add a coarse temporal target to stepwise latent prediction in HiLP?
Hierarchical Latent Prediction for Language Models
Show answer and example
Answer: Small one-step errors can compound and lose distant structure. A coarse abstract latent supervises longer spans alongside next-token and local latent objectives. This does not eliminate rollout errors or replace long-horizon tests.
Illustrative example: Predict the next waypoint and the next stage to constrain gradual route drift.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Implementation details
How isolate the effect of HiLP's coarse target?
Hierarchical Latent Prediction for Language Models
Show answer and example
Answer: Hold split, budget, and backbone fixed; ablate the block-level abstract objective and report fine and coarse errors. Synthetic diagnostics are not paper-scale LM results.
Illustrative example: Changing data and steps while adding λH confounds attribution.
Sections in the Chinese source: 核心公式, 本地复现. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Method comparison
How does coarse latent loss differ from raising the next-step weight?
Hierarchical Latent Prediction for Language Models
Show answer and example
Answer: Reweighting supervises the same horizon. A coarse target introduces a different span and structure, not simply larger λ1.
Illustrative example: Doubling next-second loss does not ask for a prediction of the next minute.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Basics
Why predict both next and abstract latents?
Hierarchical Latent Prediction for Language Models
Show answer and example
Answer: Local rollout may accumulate error; coarse targets constrain longer-term structure while preserving immediate detail.
Illustrative example: Predict the next word state with an auxiliary constraint for the next passage.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Core mechanism
Why use bilateral Kronecker curvature instead of input activation alone?
BaKron: Efficient Quantization with Kronecker-Factored Hessians
Show answer and example
Answer: Output-side curvature also changes the impact of weight error. A Kronecker approximation includes both sides in a quadratic objective, with structured parallel/divide-and-conquer rounding instead of a full Hessian. Independent rounding is not equivalent.
Illustrative example: Equal input errors can differ in loss sensitivity along different output directions.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Implementation details
Why check vectorization order and Kronecker factors together?
BaKron: Efficient Quantization with Kronecker-Factored Hessians
Show answer and example
Answer: Flattening convention determines which side each factor acts on. Swapped axes optimize a different error. Compare an explicit vectorized reference; random small weights are not checkpoint-scale evidence.
Illustrative example: Row- and column-major flattening require consistent factor conventions.
Sections in the Chinese source: 核心公式, 本地复现. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Method comparison
What does BaKron add over input-side GPTQ-style rounding?
BaKron: Efficient Quantization with Kronecker-Factored Hessians
Show answer and example
Answer: Weight vectorized error with A⊗B to include both curvatures. Optimization must match that bilateral objective; row-wise rounding is not equivalent.
Illustrative example: Lower ordinary MSE can still incur greater bilateral weighted error.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Basics
Why isn't BaKron ordinary elementwise rounding?
BaKron: Efficient Quantization with Kronecker-Factored Hessians
Show answer and example
Answer: It minimizes curvature-weighted joint error, not distance to each scalar grid point. Correlated rounding errors can amplify or cancel.
Illustrative example: Two small errors aligned in one output can be worse than a larger pair that cancels.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Core mechanism
Why doesn't AMS activation separation establish behavioral safety?
Detecting Safety Training Modification in Language Models via Activation Analysis
Show answer and example
Answer: Mid-layer contrast directions and same-architecture baseline drift provide structural evidence only. Layer numbering, variance, and baseline affect thresholds. Independent behavior evaluation is still required.
Illustrative example: A model can distinguish safety concepts internally while still responding unsafely to a particular prompt.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Implementation details
Why add one to the AMS block index in Hugging Face hidden_states?
Detecting Safety Training Modification in Language Models via Activation Analysis
Show answer and example
Answer: Block hooks are zero-based but hidden_states[0] is embeddings. Use block_id+1, select the last real token, and preserve population variance and strict threshold.
Illustrative example: Block 9 reads hidden_states[10].
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Method comparison
What do AMS Tier 1 and Tier 2 measure?
Detecting Safety Training Modification in Language Models via Activation Analysis
Show answer and example
Answer: Tier 1 measures separation of positive and negative projections. Tier 2 compares direction consistency and relative drift against matching layers/dimensions. Internal separation need not mean baseline alignment.
Illustrative example: Wide separation can coexist with a direction flip relative to baseline.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Basics
Why scan contrastive input activations?
Detecting Safety Training Modification in Language Models via Activation Analysis
Show answer and example
Answer: Identify safety-related separation in intermediate layers and compare direction/drift with a matched baseline. This does not generate or validate behavioral answers.
Illustrative example: Separable concepts do not guarantee all related requests are handled well.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why can't deleting harmful skills after writing fully replace pre-admission validation?
When Self-Evolution Backfires: Pre-Commit Gating against Skill Contamination in LLM Agents
Show answer and example
Answer: A skill can influence new trajectories and descendant skills, spreading corruption before removal. Deleting its original entry does not undo derivatives. Precommit checks structure, behavioral harmlessness, and semantic consistency before choosing a compatible set. Admission and runtime rollback are distinct. Good individual skills may conflict together.
Illustrative example: Example: An incorrect retry rule has been absorbed into another skill. Removing the original does not repair its descendant; validate before writing to reduce inherited corruption.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
Why not combine all individually high-scoring skills in the Hot set?
When Self-Evolution Backfires: Pre-Commit Gating against Skill Contamination in LLM Agents
Show answer and example
Answer: Skills can conflict or duplicate, so set utility depends on shared context. VaG selects compatible combinations by marginal gain, not the sum of independent scores. Local tests cover gates and selection, not an expensive judge or original environment. Registering discovery does not prove a promotable real controller is integrated.
Illustrative example: Example: One skill requires caching first; another clears cache every step. Each can work alone, yet loading both requires a compatibility check.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
Why keep separate structural, behavioral harmlessness, and semantic consistency gates?
When Self-Evolution Backfires: Pre-Commit Gating against Skill Contamination in LLM Agents
Show answer and example
Answer: Structure checks parsability and references, behavior checks whether known executions regress, and semantics checks intended meaning. Valid formatting can hide bad actions, and one harmless run can miss unsafe preconditions. The gates supply complementary evidence but remain limited by case coverage.
Illustrative example: Example: A perfectly formatted skill reverses 'back up, then edit.' Parsing passes it, but semantic or behavioral validation can detect the reversal.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Why is admitting experience a risky step for a self-evolving skill library?
When Self-Evolution Backfires: Pre-Commit Gating against Skill Contamination in LLM Agents
Show answer and example
Answer: Candidate experience influences later actions and can be abstracted into new skills. One bad suggestion can therefore harm more than the current task. VaG limits this propagation so continual learning does not gradually corrupt long-term memory. Write volume does not measure progress.
Illustrative example: Example: A temporary check bypass gets generalized into a rule and reused across tasks. A short-term success becomes long-term contamination.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Core mechanism
Why fix target-evaluation budget instead of comparing the highest final score alone?
HarnessOpt-Bench: Evaluating LLMs at Harness Optimization
Show answer and example
Answer: More evaluations create more search opportunities and confound optimizer quality with resources. Hidden tests isolate selection, while trusted environments and version records audit changes. Relative gains depend on baseline scale; the best visible search score is not final unseen performance.
Illustrative example: An optimizer that tries a thousand candidates versus ten is not necessarily better because its best is higher. Match call budgets first.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Implementation details
Can local candidate registration replace trusted execution and version audit?
HarnessOpt-Bench: Evaluating LLMs at Harness Optimization
Show answer and example
Answer: No. Count real budget, snapshot candidates, and isolate execution. Do not report benchmark scores without the actual model and environment. Define epsilon for relative gains near a zero baseline rather than silently changing the denominator.
Illustrative example: Adding a configuration name does not mean the optimizer changed and tested it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Method comparison
Why separate open search feedback from hidden test?
HarnessOpt-Bench: Evaluating LLMs at Harness Optimization
Show answer and example
Answer: Search uses visible feedback to choose versions, while hidden evaluation measures generalization. Repeatedly reading test during search turns it into tuning data; score gains no longer reflect unseen ability.
Illustrative example: Choosing the highest test score after dozens of variants overstates performance on unseen tasks.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Basics
What may HarnessOpt-Bench optimize, and does that include retraining the base model?
HarnessOpt-Bench: Evaluating LLMs at Harness Optimization
Show answer and example
Answer: It searches prompt, tools, control flow, and memory under a target-evaluation budget. Arbitrary base-model retraining or hidden training data are not the same search space. Record candidate versions and allowed edit scope.
Illustrative example: A search can try context compression, but replacing the base with a stronger model confounds harness gains.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why run replay verification after clustering and merging similar local skills?
Learning Globally Reusable Skills for Coding Agents
Show answer and example
Answer: Similar wording can hide different preconditions, and merging can remove a branch critical to some tasks. Replay checks whether global reuse sacrifices old behavior, while the relation graph manages conflicts. Compact clusters and new successes do not independently establish safe merges or cross-distribution transfer.
Illustrative example: Example: Merge parsers for files with and without headers, then replay both cases so the unified flow does not keep only one assumption.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
Do local relation-graph and regression-penalty tests reproduce industrial Agent F1?
Learning Globally Reusable Skills for Coding Agents
Show answer and example
Answer: No. The public mini-suite executes graph, selection, and regression operators without the paper's large model, private industrial environment, or expensive judge. Industrial F1 is full author-system evidence, not a consequence of smaller graphs or fixture scores.
Illustrative example: Example: Merging ten skills into five shows the merge happened. Without industrial test-generation tasks, it cannot imply precision or recall.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
How does global skill integration change the library differently from appending each task's experience independently?
Learning Globally Reusable Skills for Coding Agents
Show answer and example
Answer: Appending preserves local detail but accumulates similar, condition-specific, or conflicting skills. Integration uses relations to build reusable abstractions. It is not mere text deduplication: retain preconditions and validate old behavior. A smaller library or denser relations do not establish better execution.
Illustrative example: Example: Two export tasks save files, but one requires encryption and the other publication. Preserve those conditions rather than merge them into publish everything.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Why does GSE need a Skill Relation Graph for globally reusable skills?
Learning Globally Reusable Skills for Coding Agents
Show answer and example
Answer: Local experiences may overlap, complement, or conflict. The graph supports clustering and merges that avoid many near duplicates. Relationships still need evidence: semantic similarity does not make merging safe, especially when abstraction can erase environmental preconditions.
Illustrative example: Example: Both tasks run tests, but one first generates configuration and the other already has it. Preserve the difference when merging.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
Why compute group-relative advantages separately for acting and rehearsal?
Show answer and example
Answer: Action success and observation-prediction rewards have different scales and objectives. Centering them together lets one role's high scores suppress the other's learning. Separate groups let the shared policy learn environment prediction and plan with private rehearsal. Imagined observations are not tool feedback or execution evidence.
Illustrative example: A high score for predicting an open door should not mistakenly reward a failed real action; keep rehearsal distinct from the actual tool result.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
Why exclude rehearsal predictions from EnvACE's real tool-response count?
Show answer and example
Answer: Counting them would disguise imagination as execution evidence. The documented mini-suite has real_tool_responses=0; rehearsal and role-wise updates establish those paths, not browser or real-tool success.
Illustrative example: Three predicted observations are three rehearsals, not three actual API calls.
Sections in the Chinese source: 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
How does same-policy rehearsal differ architecturally from a separate world model?
Show answer and example
Answer: Rehearsal reuses the agent policy under a role condition. An independent world model has its own prediction model and training and maintenance costs. Reuse does not guarantee accuracy: keep imagined observations and real feedback separate and compute role advantages separately.
Illustrative example: 'The tool may succeed' from rehearsal is not a successful tool return.
Sections in the Chinese source: 背景与主要改动, 本地复现. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why does EnvACE switch the same policy into a rehearsal role?
Show answer and example
Answer: It predicts the next observation between real actions, performing limited private look-ahead without a separately trained world model. Training constrains acting and rehearsal; a planning prediction is not a tool execution.
Illustrative example: The agent imagines a likely response before deciding whether to take the action.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Core mechanism
Why reward code retrieval by downstream repair value rather than correct-file hits alone?
CodeGrep: An RL-Trained Retrieval Agent for LLM Coding Agents
Show answer and example
Answer: The right file may lack call chains or constraints, and retrieval may be too expensive. Pass evidence to a frozen coding agent and measure actual repair with costs. Freezing the executor reduces co-changing confounds, though one failed repair does not prove a given clue useless.
Illustrative example: Example: Finding an erroring function without its caller constraints can still lead to a bad patch. Related files and test context may enable real repair.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Implementation details
What evidence separates local retrieval operators from actual 14B GRPO training?
CodeGrep: An RL-Trained Retrieval Agent for LLM Coding Agents
Show answer and example
Answer: Local public mini-suites check bounded selection and state without training a 14B retriever, running SWE-bench, or using the full downstream fixer. Token and turn penalties need real measurements. Registration or simulated search is not executed RL or end-to-end speedup.
Illustrative example: Example: Two fewer fixture reads are a proxy reduction, not a 1.2-point improvement in actual repairs without real files and generation.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Method comparison
How does multi-turn grep, glob, and read differ from one vector Top-K?
CodeGrep: An RL-Trained Retrieval Agent for LLM Coding Agents
Show answer and example
Answer: grep and glob narrow symbols and paths; reading results informs later queries. This interactive evidence collection differs from fixing candidates by initial embedding similarity. Adaptation costs calls and context, so compare exploration resources as well as file hits.
Illustrative example: Example: Glob test directories, grep the error symbol, read a caller, then search dependencies based on actual code.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Basics
Why does CodeGrep treat evidence gathering in large repositories as a learned multi-step task?
CodeGrep: An RL-Trained Retrieval Agent for LLM Coding Agents
Show answer and example
Answer: Before repair, relevant files, symbols, and dependencies may be unknown. One search can retrieve only similar wording. CodeGrep searches, reads, and updates queries from new clues, serving downstream repair. Actual execution determines whether the evidence helped.
Illustrative example: Example: grep finds a config entry, but reading it reveals the default is elsewhere. Follow that dependency instead of guessing a local patch.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
Why can terminal answer rewards foster confirmation bias, and how does dense evidence credit help?
Contextual Information Policy Optimization for Search Agents
Show answer and example
Answer: The model may guess first and retrieve only supporting information, with an accidentally correct answer hiding poor search. Credit turns where external evidence changes later actions to reward use of information rather than call counts. Keep global outcomes so novelty does not divert attention from the question.
Illustrative example: If a search disproves a guessed date and the agent checks another source, distinguish that evidence-driven change from repeating the original guess.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
Which advantage remains at the λ=0 and λ=1 boundaries in CIPO?
Contextual Information Policy Optimization for Search Agents
Show answer and example
Answer: λ=0 uses only outcome and λ=1 only evidence, checking the mixing direction. Intermediate values combine them. Bind turn credit to real trajectory positions, and do not replace local mini-suite results with the paper's large-model scores.
Illustrative example: If evidence still changes updates at λ=0, check coefficients or a leftover branch.
Sections in the Chinese source: 核心公式, 复现边界. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
How does CIPO evidence-impact credit differ from rewarding citation counts?
Contextual Information Policy Optimization for Search Agents
Show answer and example
Answer: Citation counts measure surface form. Evidence credit concerns whether external information affects later actions. Seeing retrieved text alone does not establish causal influence; verify it under the method's definition.
Illustrative example: Ten links that change nothing may be less useful than one piece of evidence that changes a decision.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why does CIPO add evidence credit to intermediate search steps?
Contextual Information Policy Optimization for Search Agents
Show answer and example
Answer: Final answer reward does not show whether retrieval affected decisions and may encourage confirming prior beliefs. CIPO credits later action changes informed by external evidence and combines dense credit with global outcomes.
Illustrative example: Revising a plan after finding counterevidence is more informative than appending an irrelevant citation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
Why identify pivotal turns from changes between adjacent success beliefs rather than the final belief?
Show answer and example
Answer: The final value mixes all prior evidence and cannot localize the decisive turn. Accumulate teacher–student token probability differences in log odds and use the per-turn sigmoid belief increment to identify a local change. This is hindsight attribution for training; privileged replay and references stay out of independent evaluation policies.
Illustrative example: If a tool response causes belief to jump after several uneventful steps, its increment identifies the turn better than broadcasting a high final belief.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
Why accumulate AgentOPSD evidence in log odds and take sigmoid differences?
Show answer and example
Answer: Update log-odds state ℓ_t=ℓ_{t−1}+e_t and credit the difference between adjacent sigmoid beliefs. Adding e_t directly to a probability can leave its range and changes accumulation. Compute large log odds stably, and keep privileged replay out of evaluation policy inputs.
Illustrative example: The same e_t changes different prior beliefs by different amounts; it is not a fixed additive probability adjustment.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
Does critic-free AgentOPSD require no teacher information?
Show answer and example
Answer: No. It has no independent value critic, but turn evidence still uses teacher/student differences in privileged replay. Mini-suite state updates do not establish actual large-model teacher rollouts and policy training.
Illustrative example: No critic network does not remove the teacher distribution required for e_t.
Sections in the Chinese source: 背景与主要改动, 本地复现. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why does AgentOPSD use turn evidence to update success belief recursively?
Show answer and example
Answer: Trajectory rewards do not identify key decisions. Aggregate teacher/student token log-probability gaps from privileged replay by turn, accumulate them in log odds, and assign local credit from successive belief changes.
Illustrative example: A tool choice that sharply raises belief is easier to localize from the change than from an already high level.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Core mechanism
Why handle delays, lifecycle weighting, and MMoE gates together?
Show answer and example
Answer: Outcomes differ in observation time and user stage. Immediate/delayed models address labels, segments encode objectives, and MMoE shares/selects experts. Overall metrics cannot prove all groups improve.
Illustrative example: Support viewing habits for new users and deeper engagement for established users without letting immediate clicks dominate delayed payment.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Implementation details
Does local Twitch-MOR explain production gains?
Show answer and example
Answer: No. It lacks private features, mature delay labels, and traffic. Fix segments and maturity windows; never define segments using future test payments.
Illustrative example: Labeling future payers as currently high value leaks outcomes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Method comparison
Are MMoE gates and segment weights at the same level?
Show answer and example
Answer: Gates select expert mixtures per task; segment weights alter task contribution. One global weight does not implement expert interactions.
Illustrative example: New users can emphasize viewing while each task still has a different expert mixture.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Basics
Why distinguish immediate and delayed feedback in Twitch ranking?
Show answer and example
Answer: Views and interactions appear fast, while follows and payments may mature later. Lifecycle groups also value objectives differently.
Illustrative example: A new viewer who has not paid yet is not an immediate negative for value.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why does DEGR constrain same-cohort representations and adapt preference update strength to exploration reward?
DEGR: Dual Exploration-Driven Generative Re-Ranking for Adaptive Cross-Request Context Bridging
Show answer and example
Answer: High per-request scores can still expose similar items repeatedly across requests. Representation regularization reduces directional overlap, adaptive preference updates avoid reinforcing exploration without gains, and inference diversity selection acts on the final list. Spread vectors do not guarantee perceived diversity; measure relevance and exploration feedback.
Illustrative example: Similar shoes may dominate successive sensible lists. Spread candidate directions and separately verify that a new category provides worthwhile exploration.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
How should local DEGR with unchanged NDCG and lower head share be interpreted?
DEGR: Dual Exploration-Driven Generative Re-Ranking for Adaptive Cross-Request Context Bridging
Show answer and example
Answer: Logs confirm actual preference and diversity-pair updates; report ranking and exposure distribution separately. Zero NDCG gain means no main ranking improvement under this protocol, and slightly lower head share only indicates reduced concentration. Loss presence and update counts are not user value.
Illustrative example: A less popular list without better relevant positions supports wider exposure, not necessarily happier users.
Sections in the Chinese source: 本地复现. The example illustrates the mechanism.
Method comparison
Where do DEGR training diversity regularization and inference diversity-aware greedy selection act?
DEGR: Dual Exploration-Driven Generative Re-Ranking for Adaptive Cross-Request Context Bridging
Show answer and example
Answer: Squared cosine regularization spreads same-cohort embeddings; greedy selection limits redundancy relative to items already in the current list. Neither substitutes for the other. Excessive regularization separates truly similar items, and overly strong selection can hurt relevance, so ablate each.
Illustrative example: Spread embeddings do not guarantee variety on one screen; forced screen variety does not fix a representation collapsed around popular items.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why feed exploration feedback into DEGR's generator rather than just inserting random items at the end?
DEGR: Dual Exploration-Driven Generative Re-Ranking for Adaptive Cross-Request Context Bridging
Show answer and example
Answer: Random insertions collect feedback but do not teach the generator which new content is valuable. Reward-adaptive preference updates retain useful exploration, while same-cohort regularization and inference selection reduce repeats. Lower head share alone is not user benefit.
Illustrative example: Positive feedback on an uncommon outdoor item can inform future generation; random insertion alone starts from scratch each time.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Core mechanism
Why can't fast correction or slow draft-head training replace exact verification?
SpecRoll: Fast-Slow Verifier-Feedback Adaptation for Speculative Reinforcement Learning Rollouts
Show answer and example
Answer: They improve proposals as policy evolves; exact rejection/acceptance preserves target distribution. Skipping it changes RL data even with a strong drafter. No local large-model throughput evidence.
Illustrative example: Adapt the drafter after lag, but still verify every proposal.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Implementation details
Does local SpecRoll registration prove large-model RL throughput?
SpecRoll: Fast-Slow Verifier-Feedback Adaptation for Speculative Reinforcement Learning Rollouts
Show answer and example
Answer: No. Only states and selection paths are checked. Real work needs acceptance/rejection residuals, scheduling and compute measurements; listing isn't execution.
Illustrative example: A listed candidate does not demonstrate complete GPU training.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Method comparison
Does changing the SpecRoll draft change GRPO target?
SpecRoll: Fast-Slow Verifier-Feedback Adaptation for Speculative Reinforcement Learning Rollouts
Show answer and example
Answer: No. Exact target verification preserves sampling distribution, so GRPO stays unchanged. Skipping verification would change behavior data.
Illustrative example: Frequent correct drafts don't make their probabilities target policy probabilities.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Basics
Why use fast and slow draft adaptation in SpecRoll?
SpecRoll: Fast-Slow Verifier-Feedback Adaptation for Speculative Reinforcement Learning Rollouts
Show answer and example
Answer: RL changes target policy. Lightweight hidden corrections handle short drift; persistent degradation triggers future-head training. Both improve proposals without replacing target.
Illustrative example: Quickly adjust a transient acceptance drop, retrain the head only if it persists.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Core mechanism
Why optimize intervention location/method with cost instead of oversampling uncertain states?
Optimizing What Policies Learn From: Recoverability-Aware Rollout Intervention Learning
Show answer and example
Answer: Additional budget helps only recoverable states. Learn intervention-specific net gains from auditable shadow/live feedback as policy changes. Never use gold to decide; no local full environment.
Illustrative example: One more tool call can rescue a trace while no reasonable budget repairs a false premise; entropy alone can't tell.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Implementation details
What does the local RAIL path establish?
Optimizing What Policies Learn From: Recoverability-Aware Rollout Intervention Learning
Show answer and example
Answer: Reference intervention selection and cost objective, not the paper's LLMs, private environment or judges. Execute and evaluate real recoveries before capability/budget claims.
Illustrative example: Argmax over random recovery scores does not measure actual trajectory repair.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Method comparison
How does shadow-to-live evaluation differ from gold-based intervention choice?
Optimizing What Policies Learn From: Recoverability-Aware Rollout Intervention Learning
Show answer and example
Answer: Use permitted probes and actual feedback to learn recovery value. Gold would be an unavailable oracle. Deployment state must exclude training-only diagnosis.
Illustrative example: Restarting from a known solution plan is an oracle diagnostic, not a learned controller.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Basics
Why learn recoverability rather than just uncertainty in RAIL?
Optimizing What Policies Learn From: Recoverability-Aware Rollout Intervention Learning
Show answer and example
Answer: An uncertain state may be irreparable. Intervene based on expected recovery net of cost and condition on location/action, instead of uniformly spending extra.
Illustrative example: One missing tool retry and a fundamentally wrong premise have different recovery value.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Core mechanism
Why add a recoverable quantized tier between full precision and deletion?
QEvict: Recoverable Quantized KV Eviction for Attention-Drift-Robust Long-Context Decoding
Show answer and example
Answer: Attention relevance can drift. Lower storage preserves approximate content for promotion when accumulated attention rises. Permanent deletion cannot recover it; dequantization does not restore lost precision. Local state checks are not serving performance.
Illustrative example: Demote an old clue, then read it again when a later question needs it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Implementation details
What should QEvict verify when cumulative scores update?
QEvict: Recoverable Quantized KV Eviction for Attention-Drift-Robust Long-Context Decoding
Show answer and example
Answer: Verify decay γ, current attention increments, promotion threshold, and slot budget. Keep actual recoverable quantized content. A log state alone does not migrate data.
Illustrative example: Crossing τ should restore the stored vector, not just increment a count.
Sections in the Chinese source: 核心公式, 本地复现. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Method comparison
Is promotion from quantized cache equivalent to never quantizing it?
QEvict: Recoverable Quantized KV Eviction for Attention-Drift-Robust Long-Context Decoding
Show answer and example
Answer: Not necessarily. Dequantization restores access, not every original bit. Measure quantization error separately from the advantage over deletion.
Illustrative example: Converting a low-bit vector to float does not restore original decimals.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Basics
Why have full, recoverable quantized, and deleted states?
QEvict: Recoverable Quantized KV Eviction for Attention-Drift-Robust Long-Context Decoding
Show answer and example
Answer: The middle tier keeps dormant history available with less storage and supports promotion under renewed attention, unlike deletion.
Illustrative example: A previously irrelevant window can return to high-priority reading.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Core mechanism
What do shared attention and normalization versus modality-specific FFNs control?
Show answer and example
Answer: Shared components allow cross-modal interactions, while separate FFNs preserve modality-specific transformations and capacity and may reduce competition. Hold data, loss weights, and budgets fixed to identify synergy; a better aggregate does not imply equal gains for every modality.
Illustrative example: Images and text use the same attention path but separate FFNs. Evaluate each separately.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Implementation details
Why are joint objective weights part of the reproduction contract?
Show answer and example
Answer: Text, understanding, and generation terms have different gradients into shared parameters. The two coefficients alter their balance and therefore the conditions for synergy or competition. A runnable mini-suite or one perfect seed does not reproduce a 13.5B MoE trained on two trillion tokens.
Illustrative example: Increasing generation weight tenfold changes optimization pressure although loss names stay the same.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Method comparison
How does early joint training differ from concatenating separately trained modality vectors?
Show answer and example
Answer: Joint losses update shared interaction parameters and can produce cross-modal gradient synergy or competition. Concatenation only changes downstream visible features after training; it does not create shared pretraining, even at equal data volume.
Illustrative example: Updating shared attention from text and retesting vision can reveal transfer. Placing two frozen vectors side by side does not create that update.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Basics
Does modality synergy simply mean adding more data?
Show answer and example
Answer: It asks whether training on one modality improves knowledge or representations for another. Controlled experiments distinguish synergy from competition. Adding data alone changes compute, coverage, and task weighting together.
Illustrative example: If text reasoning improves after adding image training, check total tokens and compute before attributing it to shared knowledge.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
How does DBLast's shared latent address independent parallel draft positions?
DBLast: Dependent Block Drafting for Stochastic Speculative Decoding
Show answer and example
Answer: Given one latent, positions can sample in parallel; marginalizing it yields dependence across positions. Optimize expected verification length, not isolated hits. Target verification still matters and local mechanism is not service speed.
Illustrative example: One latent can coordinate singular/plural choices instead of independently picking inconsistent words.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Why check exact speculative acceptance instead of position hit rate?
DBLast: Dependent Block Drafting for Stochastic Speculative Decoding
Show answer and example
Answer: Benefit depends on accepted contiguous prefix and joint quality. Isolated correct positions need not pass as a block. Local Markov target and rank-four proposal are mechanism tests, not real throughput.
Illustrative example: Correct first and third tokens do not help the third if the second is rejected.
Sections in the Chinese source: 背景与主要改动, 本地复现. The example illustrates the mechanism.
Method comparison
How does a latent mixture differ from independent unigram proposals?
DBLast: Dependent Block Drafting for Stochastic Speculative Decoding
Show answer and example
Answer: Independent proposals multiply marginals. DBLast multiplies conditional probabilities under the same z and integrates z, so identical marginals can have different joints.
Illustrative example: Both positions can be half A/B, with only AA and BB or with all four pairs.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Basics
Why can individually likely draft positions form a poor block?
DBLast: Dependent Block Drafting for Stochastic Speculative Decoding
Show answer and example
Answer: Independent proposals ignore joint dependencies. DBLast conditions on a shared latent, inducing correlation without abandoning parallelism.
Illustrative example: Common colors and objects can still combine incoherently if chosen independently.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
Why can tasks built from initial and target states omit expert action trajectories?
State2State: Environment-Derived Mid-Training for LLM Agents
Show answer and example
Answer: The goal says where to arrive while the policy explores how. Rule-based state matching only verifies the endpoint, separating task generation from demonstrations and widening mid-training sources. Verifier-internal goals or rules are not stepwise correct-action hints. Matching must preserve task-relevant state.
Illustrative example: Given initial and target object layouts, the agent moves objects itself and the environment compares the result; it does not need a prescribed left-then-right sequence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
Why predefine the State2State projection in T(s_T)=T(s*)?
State2State: Environment-Derived Mid-Training for LLM Agents
Show answer and example
Answer: The projection determines which differences matter. Changing T after observing results changes the task. Verify initial and terminal states, legal transitions, and matching fields. A local state operator does not replace actual large-model mid-training in the paper's environment.
Illustrative example: If a goal specifies object positions, do not remove position from matching after a failure to manufacture success.
Sections in the Chinese source: 核心公式, 复现边界. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
Does target-state supervision give the same information as behavior cloning from an expert?
State2State: Environment-Derived Mid-Training for LLM Agents
Show answer and example
Answer: A target state defines success; behavior cloning also provides the action sequence. Goal supervision permits multiple valid routes. Passing verifier-computed expert paths to the policy would invalidate the no-demonstration setting.
Illustrative example: Two different paths can both succeed by reaching the goal, without copying a canonical sequence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
How does State2State construct training tasks automatically through environment exploration?
State2State: Environment-Derived Mid-Training for LLM Agents
Show answer and example
Answer: Exploration yields reachable initial and target states. A fixed projection T matches terminal state as the verifier, giving tasks without human-written descriptions or expert trajectories for every item. The policy still finds its own path.
Illustrative example: Knowing a room's initial and target layouts does not identify the sequence of actions.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
Why subtract structurally matched Observation-Ablated replay from Full replay?
Show answer and example
Answer: Comparing replay with future observations directly against the student conflates observation content and extra scaffolding. Keep the structure and remove only observations, then subtract to isolate information contribution better and amplify updates only on uncertain steps. Future information is for training attribution, not test-time input.
Illustrative example: If the replay template adds 'please check,' keep it in both versions. Only observation availability should differ so the prompt effect is not counted as observation evidence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
How do the two student log probabilities cancel in the OCSD residual?
Show answer and example
Answer: For the same token and student condition, r=(logπ_full−logπ_S)−(logπ_abl−logπ_S)=logπ_full−logπ_abl. If the student values come from different prefixes or alignments, fix matching before cancellation.
Illustrative example: Compare both expressions on the same input to detect sign or position mistakes.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
What does the Full–Ablated residual control for beyond the Full–Student gap?
Show answer and example
Answer: A single gap includes both observation content and scaffolding. A matched ablated condition removes the observation while preserving structure, so subtraction cancels shared effects. Unmatched templates leave new confounds.
Illustrative example: If Full alone includes another instruction, Full−Ablated also captures that instruction's effect.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why does OCSD worry that replay scaffolding itself changes token scores?
Show answer and example
Answer: Extra structure and prompt formatting during future-observation replay can affect predictions independently of the observation. Use structurally matched Full and Observation-Ablated controls to extract a residual better focused on the observation.
Illustrative example: A long replay template can change probabilities, so not every change comes from environmental evidence.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why does Quality per Cost require more than shortening a query summary?
MemoryCPT: An End-to-End Agent Memory Framework for Cost-Performance Trade-off
Show answer and example
Answer: Cost savings can harm answer quality, so the ratio may not improve. Construction, retrieval, and summarization each have inference costs. Offline chain distillation reduces construction, RRF and query-focused summaries manage reads, and cost-aware reward judges the pipeline. Compare the same quality definition rather than call fewer tokens success.
Illustrative example: Example: Reducing 500 words to twenty may omit a crucial exception and cause a wrong answer. Eighty useful words may offer a better quality-cost trade-off.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
Can text concatenation stand in for RRF retrieval and a LoRA summarizer?
MemoryCPT: An End-to-End Agent Memory Framework for Cost-Performance Trade-off
Show answer and example
Answer: No. RRF actually fuses rankings, the summarizer generates query-relevant context, and cost-aware GRPO needs real rewards and updates. Local tests retain core selection or state diagnostics without paper-scale training or benchmarks. Interface fields are not completed distillation or RL.
Illustrative example: Example: Concatenating two ranked lists performs neither reciprocal rank fusion nor learned summarization. Supplying the text to a model does not mean QAR ran.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
Why optimize construction cost and per-query cost separately?
MemoryCPT: An End-to-End Agent Memory Framework for Cost-Performance Trade-off
Show answer and example
Answer: Construction processes incoming history, and reads recur with queries. They have different frequencies and useful information. MemoryCPT distills offline chains for construction and applies rank fusion and query-related summaries for reads. Sum actual write and query frequencies and check answer quality before claiming total savings.
Illustrative example: Example: Thousands of turns written but two queries can make construction dominate. Hundreds of reads from one history can compound small per-read savings.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Why account separately for memory-writing and memory-use costs in long-lived assistants?
MemoryCPT: An End-to-End Agent Memory Framework for Cost-Performance Trade-off
Show answer and example
Answer: History is written continually, while repeated queries trigger retrieval and summaries. Frequencies and cost drivers differ. MemoryCPT studies the end-to-end trade-off, making construction cheaper and reads question-specific. Shortening one summary does not establish system efficiency.
Illustrative example: Example: Hundreds of daily writes and five reads make different bottlenecks than five writes and hundreds of reads.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
Why make Belief, Progress, and Experience external state the policy can read and write?
EvoHarness-RL: Learning Self-Evolving Runtime Harness for Long-Horizon LLM Agents
Show answer and example
Answer: A fixed harness can make inappropriate decisions about storage and merging. Explicit actions teach the policy to manage belief, progress, and experience, using SFT to learn the interface and then optimizing task reward minus management cost. More writing is not inherently useful, and state must reflect actual observations.
Illustrative example: Write new constraints to Belief and merge duplicates into Experience. Appending every log each turn can consume more than the task gains.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
Why bind EvoHarness-RL's harness cost to actual execution logs?
EvoHarness-RL: Learning Self-Evolving Runtime Harness for Long-Horizon LLM Agents
Show answer and example
Answer: In R=R_task−λC_harness, cost must reflect executed reads, writes, and merges, not proposals alone. Check whether later policy steps read the updated state. The local mini-suite is not full paper-scale LLM RL.
Illustrative example: Text saying 'merge experience' without a state update is not a successful harness action.
Sections in the Chinese source: 核心公式, 复现边界. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
How does a learned harness policy differ from configuring fixed memory?
EvoHarness-RL: Learning Self-Evolving Runtime Harness for Long-Horizon LLM Agents
Show answer and example
Answer: A fixed module prescribes read/write behavior. A learned harness policy chooses when to read, write, or merge and pays operation costs. An 'evolvable' registry label on a fixed pipeline does not establish learned external-state management.
Illustrative example: Saving all history every turn is different from deciding whether to save based on current progress.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why does EvoHarness-RL make runtime-state reads and writes trainable actions?
EvoHarness-RL: Learning Self-Evolving Runtime Harness for Long-Horizon LLM Agents
Show answer and example
Answer: If fixed scripts maintain Belief, Progress, and Experience, the policy cannot learn when updates or merging are worthwhile. SFT teaches harness actions, then task rewards and costs train when to use them.
Illustrative example: A simple task may need few experience edits, while a long task benefits from saving intermediate progress.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Core mechanism
Why is prior strength κ as important as the probability p when turning an LLM judgment into Beta pseudocounts?
LLM-Derived Priors for Thompson Sampling in Cold-Start Comment Recommendation
Show answer and example
Answer: p sets the initial preference and κ controls how much real feedback changes it. A large κ can let wrong semantics dominate for too long, while a tiny one barely helps cold start. Thompson sampling explores and updates the posterior, so the LLM is a starting point. A grouped local proxy does not establish general usefulness of text priors.
Illustrative example: Illustration: Two items start at 80%, but one prior represents a few observations and the other thousands. Repeated failures overturn the first much sooner.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Implementation details
Why distinguish prior pseudocounts from observed successes and failures when updating a Beta prior?
LLM-Derived Priors for Thompson Sampling in Cold-Start Comment Recommendation
Show answer and example
Answer: Initialize parameters as 1+κp and 1+κ(1−p). Add counts only for arms actually shown and observed, and keep groups separate as specified. Future test feedback cannot initialize the prior. The local genre proxy does not prove actual LLM semantic calls.
Illustrative example: Illustration: An unselected arm has no outcome this round. Counting it as a failure because another arm won distorts the posterior and suppresses exploration.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Method comparison
Why do equal p and unequal κ produce different responses to early feedback?
LLM-Derived Priors for Thompson Sampling in Cold-Start Comment Recommendation
Show answer and example
Answer: p determines the mean, while κ determines how much pseudo-evidence supports it. A stronger prior resists real observations, changing exploration even at the same mean. Too strong can lock in errors; too weak loses useful knowledge. No κ is universally better for all groups.
Illustrative example: Illustration: Two 60% priors represent two and 200 observations. Three failures change the former far more.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Basics
Why turn LLM judgments into Beta pseudocounts instead of making direct recommendations in LLM-TS Prior?
LLM-Derived Priors for Thompson Sampling in Cold-Start Comment Recommendation
Show answer and example
Answer: Pseudocounts encode an initial success probability in a Bayesian state that feedback can update. Thompson sampling balances exploration, and real observations should be able to override the prior. Text can be biased and user clusters need not be homogeneous.
Illustrative example: Illustration: A favorable text judgment supplies a moderate prior for one group. Later clicks and skips correct it instead of the system recommending the item forever.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Core mechanism
Why train reward-specialized teachers and merge on student states?
SMOPD: Multi-Reward Reinforcement Learning via Specialize-and-Merge Online Policy Distillation
Show answer and example
Answer: Separate rewards produce different strengths, then weighted distillation uses their distributions where students act. This isn't averaging rewards into one teacher or merging weights, and abilities can conflict. Local candidate evidence only.
Illustrative example: Calculation and formatting specialists can both score one student reasoning step.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Implementation details
Which states should all SMOPD teacher KL terms align to?
SMOPD: Multi-Reward Reinforcement Learning via Specialize-and-Merge Online Policy Distillation
Show answer and example
Answer: Score identical student prefixes, keeping weights/teacher targets frozen. Independent teacher generations do not align positionwise. No real local specialist training/merge.
Illustrative example: One teacher's step two and another's step four are not shared supervision states.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Method comparison
How do SMOPD reward specialists differ from MOPD domain specialists?
SMOPD: Multi-Reward Reinforcement Learning via Specialize-and-Merge Online Policy Distillation
Show answer and example
Answer: SMOPD varies reward priorities; MOPD often divides task domains. Both use multiple teachers, but expert grouping/evaluation and budgets differ.
Illustrative example: One domain can have several reward specialists, not one expert per dataset.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Basics
Why specialize then merge in SMOPD?
SMOPD: Multi-Reward Reinforcement Learning via Specialize-and-Merge Online Policy Distillation
Show answer and example
Answer: Reward priorities produce complementary teachers. Merge dense supervision on student-visited states instead of forcing conflicting goals into one expert initially. Teachers can still interfere.
Illustrative example: Train rigor and format specialists before deciding how each teaches the student.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Core mechanism
Why locate the earliest wrong step and use short-horizon repair?
CausalOPD: First-Wrong-Step Supervision for Distilling Causal Chain Reasoning
Show answer and example
Answer: Later mistakes can inherit an earlier false premise. Correct the turning point rather than imitate final conclusions, then organize evidence/mechanism/conclusion. The teacher can misidentify it; proxy diagnostics aren't causal reasoning capability.
Illustrative example: Repair the first confusion of correlation with causation, not just the last sentence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Implementation details
May first-error search default to index zero if nothing is found?
CausalOPD: First-Wrong-Step Supervision for Distilling Causal Chain Reasoning
Show answer and example
Answer: No. Explicitly represent no error or unreliable judgment. Align step indices and keep privileged diagnosis out of inference. Local candidates don't implement the full causal teacher.
Illustrative example: An empty list must not invent a first-step error.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Method comparison
What tradeoff separates short first-error repairs and whole-trajectory imitation?
CausalOPD: First-Wrong-Step Supervision for Distilling Causal Chain Reasoning
Show answer and example
Answer: Short windows focus local breaks and reduce irrelevant supervision; full imitation covers more but may diverge from student behavior. Incorrect localization can miss the real error.
Illustrative example: Fixing one mechanism link doesn't guarantee every conclusion is correct.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Basics
Why prioritize the first error in CausalOPD?
CausalOPD: First-Wrong-Step Supervision for Distilling Causal Chain Reasoning
Show answer and example
Answer: Later errors can follow from an early break. Short repair at its source avoids redundant punishment, then a curriculum moves evidence→mechanism→conclusion. Teacher localization must be reliable.
Illustrative example: An incorrect premise can make later coherent derivation wrong.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Core mechanism
Why check teacher confidence against realized return before using token guidance?
Show answer and example
Answer: Privileged certainty can amplify error. Normalize within interaction steps, gate by positive return association and use the existing reward-to-advantage path. Negative association isn't help; inference does not use privileged training skill/outcome.
Illustrative example: Gate out confidence in fluent but ineffective tool calls.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Implementation details
Does ADRS standardize over a full trajectory or within interaction steps?
Show answer and example
Answer: Standardize teacher scores within each interaction step and correlate across trajectory returns, then feed host reward-to-advantage logic. Local z-scores aren't correctness labels.
Illustrative example: Compare second tool calls with other second calls, not every turn combined.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Method comparison
How does TVA correlation gating differ from fixed distillation weight?
Show answer and example
Answer: Fixed weight always trusts teacher; positive correlation retains signals associated with returns. Batch correlation is not causality and may not use test gold.
Illustrative example: If teacher scores don't track success, extra guidance weakens.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Basics
Why not simply trust high privileged-teacher confidence in ADRS?
Show answer and example
Answer: Extra skills/context may boost confidence without returns. Gate dense feedback by correlation between same-step teacher scores and observed returns.
Illustrative example: A teacher can favor a tool action that frequently fails.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Core mechanism
Why use task-guide attention if TaskPress is query agnostic?
TaskPress: Query-Agnostic KV Cache Compression via Task-Guided Pruning
Show answer and example
Answer: Guide attention supplies reusable task-level preference, combined with value scale, without the actual downstream query. Discard guide KV and retain historical positions. A small kernel does not validate guide generation or multitask gains; full run is not reproduced due to cost.
Illustrative example: Summarization guidance can compress text before a future question; reading that question first changes the task.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Implementation details
Can edge average pooling divide by valid neighbors?
TaskPress: Query-Agnostic KV Cache Compression via Task-Guided Pruning
Show answer and example
Answer: No. Use odd K, zero padding, and fixed divisor K. Keep original position IDs after gathering, discard guide KV, and note low-index tie-breaking is local.
Illustrative example: At the first token with K=3, still divide by three even if only two neighbors are valid.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Method comparison
Why multiply key importance and value scale?
TaskPress: Query-Agnostic KV Cache Compression via Task-Guided Pruning
Show answer and example
Answer: Guide attention measures read frequency and value scale captures amplitude relevant to quantization. The product accounts for matching and content; either term alone changes the method.
Illustrative example: Equal key importance can produce different scores at different value magnitudes.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Basics
Does query agnostic mean no task information?
TaskPress: Query-Agnostic KV Cache Compression via Task-Guided Pruning
Show answer and example
Answer: No. Use guide-to-context attention but not the actual query. Task conditioning differs from query-specific selection.
Illustrative example: Know the task is summarization without knowing the exact future question.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why check memory actions with separate local and global verifiers?
Show answer and example
Answer: Local checks validate one transition contract but cannot judge consistency across a long trajectory. Global verification checks the relationship between the whole trace and final memory. Both complement task reward so legal individual steps cannot quietly accumulate into false state. Verifier pass rate is not task accuracy.
Illustrative example: Example: Deleting a field may be locally valid but remove the only evidence for a later answer. Global evidence consistency should catch that.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
Does nonzero local operation telemetry reproduce all seven operations and full RL?
Show answer and example
Answer: No. The README says the local run maintains three states and executes core retrieve, add, restore, and related paths. Counts prove those branches ran, not original multi-GPU RL or full environments. Claims must follow actual operations and budgets, not method names or counts.
Illustrative example: Example: A positive restore count means recovery was invoked, not that a learned policy picks the right time on real long tasks.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
Why do final task reward and memory-operation checks supervise different things?
Show answer and example
Answer: Final reward measures the overall result but cannot locate a retrieval, storage, or restore fault. Verification checks transitions and evidence, while operation cost controls resource use. Success can contain illegal or wasteful steps, and legal steps can fail to solve the task. Keep the signals distinct.
Illustrative example: Example: A lucky correct answer does not justify deleting key evidence. A legal retrieval need not find the answer.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Why manage long-term memory, active context, and episodic history together?
Show answer and example
Answer: They jointly decide where information lives and what is visible. Separate optimization can misalign writing, restoring, and reading. VerMem selects atomic operations with one policy and verifies transitions locally and evidence globally. A shared policy does not make the state types interchangeable.
Illustrative example: Example: A fact in long-term memory needs retrieve or restore before answering. Merely existing in history does not make it visible.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Core mechanism
Why does lifting a trajectory into a function workflow graph help reuse it with new tools?
Show answer and example
Answer: The graph preserves required operations and dependencies while deferring concrete tools and interfaces until execution. A new tool set can implement the same functional plan using its current schemas. Parameter contracts still matter; each node needs a real tool with the required input/output behavior.
Illustrative example: Example: Old search_web and new lookup can implement evidence retrieval, so the workflow can be reused, but arguments and response formats must be matched again.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Implementation details
Why preserve state dependencies in Lift and Select rather than saving only a list of functions?
Show answer and example
Answer: Edges show what evidence or object each later step needs, and Select chooses tools based on current state. An unordered list loses variable provenance and preconditions. The mini-suite verifies lifting and selection control flow without real tools; perfect fixture scores do not establish transfer to unseen services.
Illustrative example: Example: Retrieve an order_id before canceling that order. A list saying query and cancel without binding the ID cannot produce a legal second call.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Method comparison
What information does a function workflow graph need versus an executable tool plan?
Show answer and example
Answer: The function graph describes desired operations and state dependencies, preserving procedure across catalogs. An executable plan must also bind each node to a current API, parameters, and actual objects. The first gives transferable abstraction and the second grounds it. A sensible graph does not guarantee valid arguments, and legal calls need not cover the whole task.
Illustrative example: Example: The graph says find customer then find orders. Execution must confirm whether the new service returns customer_id or account_id and which field the order API accepts.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Basics
What experience does ToolLIFT transfer when tool sets change frequently?
Show answer and example
Answer: API names, arguments, and response formats change across services, but functional steps can stay stable, such as checking identity before reading records. ToolLIFT retains that procedural structure and resolves it against the current tool catalog rather than discarding old traces. Concrete interfaces must still be checked.
Illustrative example: Example: A migration still requires finding a customer before retrieving orders, but the old function names and fields cannot be copied into the new service.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why use separate wake and send agents plus a filter?
A Self-Triggered Agentic Push Recommendation System
Show answer and example
Answer: Ordinal planning decides when to observe, trajectory reward drives sending, and a light filter rejects low-value or abnormal plans. This combines observation cadence and actions, without equating waiting value to message value or local ranking with retention.
Illustrative example: Delay after an ignored notification, then still decline sending; filter one-second wake loops.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Does local STEPS have real permissions and wall-clock wakeups?
A Self-Triggered Agentic Push Recommendation System
Show answer and example
Answer: No. MovieLens verifies decision decomposition, not push permissions, reach cost, clocks, or delivery. Actual service needs contracts and real rewards.
Illustrative example: Updating a simulated timestamp does not notify a phone.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
Are planning, execution, and filtering one threshold?
A Self-Triggered Agentic Push Recommendation System
Show answer and example
Answer: No. Planning predicts wake interval, execution decides send, and filtering controls cost and anomalous plans. Combining them into one probability removes scheduling and guardrails.
Illustrative example: Choosing tomorrow morning differs from choosing to send now.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why decide both whether to push and when to wake again?
A Self-Triggered Agentic Push Recommendation System
Show answer and example
Answer: Fixed frequency ignores user state; polling wastes work. Planning, execution, and cheap filters jointly handle timing and cost.
Illustrative example: A no-send decision should still specify when to reevaluate.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Core mechanism
Why multiply rewrite confidence and item relevance while separating embedding gradients?
SPEAR: Selection-aware Personalized End-to-end Adaptive Rewriting and Retrieval for Community Search
Show answer and example
Answer: Generic rewrites can win with misleading path scores. Product gating needs both trust and relevance; gradient isolation protects original retrieval semantics from ranking optimization. Request-level calibration does not guarantee correct confidence.
Illustrative example: Replacing waterproof hiking boots with “popular products” should lose confidence despite high clicks.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Implementation details
Does the local two-path SPEAR proxy generate real rewrites?
SPEAR: Selection-aware Personalized End-to-end Adaptive Rewriting and Retrieval for Community Search
Show answer and example
Answer: No. MovieLens content proxies products/posts without private sessions or a production generator. Keep original scores and isolate embedding gradients; do not pick rewrites on test.
Illustrative example: Scoring a handwritten rewrite does not show generation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Method comparison
How does a multiplicative gate differ from adding confidence and relevance?
SPEAR: Selection-aware Personalized End-to-end Adaptive Rewriting and Retrieval for Community Search
Show answer and example
Answer: Both inputs must support the product; addition can let one high score compensate for the other. Dynamic selection also calibrates request-level scale and bias.
Illustrative example: A confident rewrite cannot rescue an irrelevant item by confidence alone.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Basics
Why preserve original-query retrieval semantics in SPEAR?
SPEAR: Selection-aware Personalized End-to-end Adaptive Rewriting and Retrieval for Community Search
Show answer and example
Answer: Generic rewrites can exploit high path scores while drifting. Separate embeddings and gradients preserve retrieval; confidence times item relevance limits amplification.
Illustrative example: A running-shoe query should not rank unrelated sports items by popularity alone.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why does KGD select related future items and make the transfer residual orthogonal to the knowledge anchor?
Knowledge–Geometry Decoupling: Refreshable Pretrained Transfer for Streaming Recommendation
Show answer and example
Answer: An immediately adjacent click can be noise or a task switch. BMTP picks collaboratively or semantically related future targets to focus pretraining. During transfer, the refreshable knowledge encoder is frozen for the task to read; the residual removes its projection onto the anchor to add new geometry without directly overwriting learned knowledge. Orthogonality constrains direction, not lossless coexistence of all task information.
Illustrative example: Illustration: An adjacent accidental click should not provide the same supervision as a persistent photography interest. A new task can try to add price sensitivity outside that anchor rather than rewriting the existing photography semantics.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
How should KGD verify gradient flow and zero-norm handling for read-only knowledge and orthogonal projection?
Knowledge–Geometry Decoupling: Refreshable Pretrained Transfer for Streaming Recommendation
Show answer and example
Answer: Freeze the knowledge encoder or explicitly version its refreshes; task loss must not update it through the read-only interface. ACR's projection denominator depends on anchor norm, so define handling for zero or near-zero anchors. Merely subtracting r−e is not orthogonal projection. The bounded local mechanism does not reproduce full streaming production transfer.
Illustrative example: Illustration: A zero anchor has no direction, so division by its norm is invalid. If knowledge parameters change after backward, the transfer is not read-only even if output shapes look right.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
What kind of overwrite does KGD's orthogonal ACR restrict relative to an ordinary residual adapter?
Knowledge–Geometry Decoupling: Refreshable Pretrained Transfer for Streaming Recommendation
Show answer and example
Answer: ACR removes the residual projection along the knowledge anchor to avoid directly changing the existing main direction. An ordinary residual can add in any direction, increasing flexibility and the risk of interference. Orthogonality is a geometric constraint, not proof that knowledge is unchanged or semantics independent. Verify effects by task.
Illustrative example: Illustration: If the anchor already points toward photography, an adapter can add budget distinctions instead of only amplifying that direction. A perpendicular vector could still contain noise.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why does KGD's BMTP select related future items rather than supervise all adjacent clicks equally?
Knowledge–Geometry Decoupling: Refreshable Pretrained Transfer for Streaming Recommendation
Show answer and example
Answer: Adjacent events can be accidental or express different needs; predicting them blindly encodes unhelpful transitions. BMTP selects more meaningful future targets by collaborative or semantic relations and later exposes the knowledge as read-only. Bad relations can miss real emerging interests, and future labels remain training supervision only.
Illustrative example: Illustration: An accidental sports-news click after camera browsing need not define a strong transition. A related future lens looks more like the same need, but that is not a reason to exclude all cross-topic exploration.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why are DME's reasoning and reconstruction heads used only during training?
Douyin Multimodal Embedding Model Technical Report
Show answer and example
Answer: Contrastive retrieval can lose fine-grained evidence. Typed latent targets and cross-reconstruction push embeddings to retain relevant information about the other side. Serving uses efficient similarity without per-pair reasoning text; auxiliary losses do not guarantee faithful explainability.
Illustrative example: Teach query vectors to retain relevant material and use cases, then serve nearest-neighbor retrieval without generating explanations.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
Why verify that DME's generation heads are removed from serving?
Douyin Multimodal Embedding Model Technical Report
Show answer and example
Answer: Both heads provide training supervision only. Running reconstruction online violates intended latency and dependencies; never training the heads does not establish reconstruction learning. The local ridge diagnostic is distinct from the paper's large multimodal heads.
Illustrative example: Log both reconstruction losses and export only the two encoders. Turning heads on for quality and off for latency would evaluate different models.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
What does DME typed latent evidence retain or give up compared with online chain of thought?
Douyin Multimodal Embedding Model Technical Report
Show answer and example
Answer: Structured internal evidence trains embeddings while serving stays vector-based. Online CoT provides readable steps at extra latency. DME trades that textual explanation for retrieval efficiency, and latent evidence does not establish faithful chains. Local reconstruction proxies are not full multimodal heads.
Illustrative example: Train with place and purpose evidence types but serve retrieval only, without claiming the system generated a stepwise rationale.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why does DME use cross-conditioned reconstruction as well as contrastive learning?
Douyin Multimodal Embedding Model Technical Report
Show answer and example
Answer: Contrastive supervision tells embeddings which examples should be near but can lose distinctions. Reconstructing document details from queries and query details from documents encourages compressed embeddings to keep useful semantics, not lossless reconstruction or extra serving information.
Illustrative example: Two camping documents share a theme but differ in winter warmth versus light weight; reconstruction encourages preserving that difference.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Core mechanism
Why accumulate sustained teacher-favoring evidence rather than switch on single gaps?
Show answer and example
Answer: Single differences are noisy while whole-response weights hide position. Windowed exponential accumulation and decline decay combine smooth OPSD/GRPO gates. Persistence stabilizes signals without proving teacher correctness.
Illustrative example: An isolated overestimated word need not receive the same supervision as a consistently supported passage.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Implementation details
May PCSD accumulation continue across independent responses?
Show answer and example
Answer: No. Reset per response and apply valid masks. Teacher/gate statistics are fixed supervision; deterministic checks aren't long-reasoning training.
Illustrative example: Previous problem's ending support cannot become the next one's prior.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Method comparison
How does PCSD continuous gating differ from a hard threshold?
Show answer and example
Answer: Sigmoid varies strength smoothly; accumulated trends allow the same present gap to receive different weights. It still relies on teacher quality, not a new verifier.
Illustrative example: Equal current gaps can differ if one has risen consistently and the other fallen.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Basics
Why use persistent signals rather than one token gap in PCSD?
Show answer and example
Answer: Local gaps are noisy; one whole-response weight is coarse. Accumulate support in adaptive windows, decay declining trends and continuously gate supervision.
Illustrative example: One high teacher score should differ from several supported consecutive steps.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
Why not normalize native and external rollouts together?
Show answer and example
Answer: Differences in length and rewards can turn provenance into advantage, letting one source dominate. ERILS caps external lengths and treats reward sources separately. This preserves exploration value, not immunity to off-policy errors.
Illustrative example: If outsiders are always longer and higher-scoring, pooled normalization may make every student sample negative.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
Why explicitly enforce external response length in ERILS?
Show answer and example
Answer: Long trajectories consume tokens and change loss scale. Retain length constraints and source labels through normalization. Candidate-policy diagnostics do not reproduce diffusion checkpoint training.
Illustrative example: An oversized expert trajectory could consume the batch and change the intended mixture.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
Why normalize sources separately rather than mix on/off-policy rewards?
Show answer and example
Answer: Sources can differ in reward scale and distribution; pooled baselines can systematically suppress one. Separate normalization preserves within-source credit without solving every off-policy bias.
Illustrative example: A higher expert mean can make useful student improvements negative under one pooled baseline.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
Why introduce external rollouts in ERILS diffusion RL?
Show answer and example
Answer: Student exploration may lack coverage, while external policies offer different denoising trajectories. Track source and length; high external reward isn't native reachability.
Illustrative example: Log an external model's alternative denoising path as external.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Core mechanism
What does a short-window TTCD student learn beyond final answers?
Learning What to Remember: Test-Time Training via Context Distillation
Show answer and example
Answer: Long-window teacher hidden states supervise fast weights to retain context useful for future predictions. The student must select information under finite memory, not uniformly preserve every token. State distillation is not lossless.
Illustrative example: Store an early variable definition in fast weights to match later teacher state, rather than memorizing one answer.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Implementation details
Why align teacher and student positions and representations?
Learning What to Remember: Test-Time Training via Context Distillation
Show answer and example
Answer: Hidden loss compares the same t with compatible dimensions. Offsets or accidental broadcasting optimize the wrong target. Local results are not paper-scale long-context validation.
Illustrative example: Student step 20 aligns to teacher step 20, not the final document state.
Sections in the Chinese source: 核心公式, 复现边界. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Method comparison
What does state distillation constrain beyond final token matching?
Learning What to Remember: Test-Time Training via Context Distillation
Show answer and example
Answer: It aligns intermediate teacher and student representations to train memory, not just output probabilities. Supervision location and space differ.
Illustrative example: Two students can predict the same token while retaining different historical information.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Basics
What information differs between teacher and student?
Learning What to Remember: Test-Time Training via Context Distillation
Show answer and example
Answer: Teacher sees longer history, while student relies on its window or fast weights. Hidden-state targets teach retention without giving deployed students full history.
Illustrative example: A teacher reads background; deployment still uses finite learned memory.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Core mechanism
Why add a separate depth route for V but retain shared Q/K routing?
Role-Decoupled Attention Residuals
Show answer and example
Answer: Q/K determine matching while V supplies retrieved content. RD-AttnRes retains the source set and attention mechanism, but learns separate depth weights for content. Routing does not create new source information.
Illustrative example: Q/K can favor deep semantics while V preserves shallow lexical details.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Implementation details
Should Q and K also get separate routes?
Role-Decoupled Attention Residuals
Show answer and example
Answer: No. Both use h_QK, with h_V separate. Splitting Q/K adds another variable. Check actual weight divergence without treating it as guaranteed PPL improvement.
Illustrative example: Q and K project from one mixed representation, while V projects from another.
Sections in the Chinese source: 核心公式, 本地复现. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Method comparison
What does the Block AttnRes comparison hold fixed?
Role-Decoupled Attention Residuals
Show answer and example
Answer: Residual sources and main attention stay the same; only V gets an independent depth route. This isolates role separation rather than changing the backbone.
Illustrative example: Both arms read the same layer set, with one allowing different V weights.
Sections in the Chinese source: 背景与主要改动, 本地复现. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Basics
Does depth routing choose tokens or historical layer representations?
Role-Decoupled Attention Residuals
Show answer and example
Answer: It combines residual sources from different depths, separately for QK and V, not token-level experts.
Illustrative example: Use deeper matching features for QK and shallower content for V.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Core mechanism
Why keep recurrent states from several chunks instead of one final DART state?
DART: Decoded Attention over Recurrent States for Efficient Long-Context Sequence Modeling
Show answer and example
Answer: Fixed-capacity recurrence can overwrite old facts. DART decodes chunk-state contributions into queryable K/V, attends to them from the current token, and fuses memory reads with Mamba. Addressability costs extra storage and retrieval.
Illustrative example: A later chapter may overwrite an earlier author's name; retrieve the first chunk instead of relying on the final state.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Implementation details
Which path should DART's gate g_t modulate?
DART: Decoded Attention over Recurrent States for Efficient Long-Context Sequence Modeling
Show answer and example
Answer: Mamba is the main output; add gated state-memory attention. Separate recurrent updates, state-to-K/V decoding, and query reads. Do not gate the entire sum.
Illustrative example: Near-zero g_t suppresses memory reads while preserving Mamba.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Method comparison
How does DART state memory differ from storing all token KV?
DART: Decoded Attention over Recurrent States for Efficient Long-Context Sequence Modeling
Show answer and example
Answer: It reads K/V decoded from compressed recurrent chunks, not original K/V for every token. This trades storage and access flexibility and can still lose detail.
Illustrative example: Retrieving a chunk representation need not reconstruct every word.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Basics
Why make recurrent state queryable rather than only use it as a final summary?
DART: Decoded Attention over Recurrent States for Efficient Long-Context Sequence Modeling
Show answer and example
Answer: Continuous updates can erase old information. DART retains chunk contributions for targeted access, giving long-range dependencies an addressable path.
Illustrative example: Retrieve chapter one's state to answer a name question after many chapters.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
If a judge can read gold to generate critiques, why can't that critique be supplied directly during search evaluation?
HindSearch: Trajectory-Level Hindsight Critique for Search-Augmented Reinforcement Learning
Show answer and example
Answer: Gold helps the training judge explain failures and turn sparse outcomes into actionable distillation signals. A reference-derived critique for the same evaluation task indirectly discloses answer information, invalidating independent search. Freezing the judge does not remove privileges in its inputs.
Illustrative example: Training feedback can use the correct date to identify a wrong source, but testing must provide only the question, not a critique revealing that Wednesday is correct.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
Who can read gold in HindSearch, and which inputs must stay out of independent search evaluation?
HindSearch: Trajectory-Level Hindsight Critique for Search-Augmented Reinforcement Learning
Show answer and example
Answer: Only the frozen training judge and independent scoring side get gold. Critique derived from it is privileged training feedback. The evaluated policy cannot receive the current item's gold, judge object, or gold-derived hints without contaminating success rates.
Illustrative example: Placing a judge's answer-based corrective suggestion in a test search prompt is not leakage-free inference.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
What does a trajectory-level hindsight critique add beyond binary success or failure values?
HindSearch: Trajectory-Level Hindsight Critique for Search-Augmented Reinforcement Learning
Show answer and example
Answer: Critique can identify specific shortcomings and their relationships, while binary outcomes only show success. A teacher or judge produces it at a cost and may be wrong; longer explanations are not automatically more accurate supervision.
Illustrative example: Two zero-scoring traces could reflect missing retrieval versus misread evidence, which a single zero does not distinguish.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why does HindSearch generate hindsight critique after a failed search trace completes?
HindSearch: Trajectory-Level Hindsight Critique for Search-Augmented Reinforcement Learning
Show answer and example
Answer: Terminal success is sparse. A frozen judge can use gold to identify problems in a completed failure and supply auxiliary on-policy distillation signals. Hindsight explains training failures; it is not future answer information available during evaluation.
Illustrative example: After training execution, note that search ignored key evidence to improve the next policy rather than answer the current test item.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why alternate freezing the router and memory bank rather than updating both freely each round?
Show answer and example
Answer: Routing changes which memories are seen; bank changes alter router feedback. If both drift together, effects are hard to attribute. Freeze one while updating the other to limit nonstationarity, and use a lightweight residual to preserve the base LLM prior. This makes feedback easier to learn and audit without guaranteeing convergence or gains.
Illustrative example: Example: Freeze memory to compare routes, then freeze routing to revise low-value entries. Changing both at once makes a failure hard to locate.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
Why include task axis and tool signature in a retrieval key instead of reusing by query text alone?
Show answer and example
Answer: Similar queries may need different routes for different tasks or tool versions. Axis and signature prevent reuse across incompatible contracts. The local run checks rewrite, residual, and alternating update flow. A public mini-suite and Evolve registry entry do not establish full learning or all paper benchmarks.
Illustrative example: Example: Two order lookup services return different schemas. Matching query words do not justify a memory route that uses obsolete fields.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
What are the roles of the frozen LLM's routing prior and the trainable residual router?
Show answer and example
Answer: The base LLM contributes task understanding as a routing prior. A residual changes selection using real interaction feedback without retraining the full model. The prior bounds updates, while the residual adapts to current memory. It learns from executed routes only and is not guaranteed to beat the prior.
Illustrative example: Example: A base route favors generic retrieval, but the residual learns from current tool tasks to favor an experience store. The language model's weights stay frozen.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Why do routing policy and memory-bank updates affect each other?
Show answer and example
Answer: Routing determines which memories are used; outcomes update route and memory utility. Content and graph changes then change route value. Optimizing one side alone can preserve the other's defects. CoEvo-Mem alternates updates and learns only from actual trajectories, not invented results for untried routes.
Illustrative example: Example: A route keeps selecting stale records. Changing the bank alters its payoff; after the update, routing must learn which paths work.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why does HRPO split the final reward into prefix residual credit and credit-to-go?
Hierarchical Residual Policy Optimization for Generative Recommendations
Show answer and example
Answer: Broadcasting a whole-item reward to each SID token assigns identical responsibility to distinct decisions and can raise variance. HRPO smooths prefix utilities within user clusters, takes differences between neighboring prefixes as local credit, and accumulates subsequent credit so earlier decisions can learn their longer-term effects. Prefix utility is estimated: the decomposition does not automatically remove bias, and inference cannot read the true target path.
Illustrative example: Illustration: The first code can choose the wrong category and doom the entire path, while the last only refines a choice within the category. Prefix credit tries to distinguish those roles rather than treating all codes alike.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
How can a telescoping check verify HRPO's prefix differences?
Hierarchical Residual Policy Optimization for Generative Recommendations
Show answer and example
Answer: Adjacent deltas should use the same version of utility. Their sum should equal final-prefix utility minus empty-prefix utility, and credit-to-go sums from the current position, not the start. User-cluster smoothing must not include future test feedback. The local binary SID verifies this bounded mechanism, not large-scale generative post-training performance.
Illustrative example: Illustration: If the empty-prefix utility is zero and the next three utilities are 1, 3, and 2, the deltas are 1, 2, and −1, which sum to 2. A different total from mixed smoothing tables exposes an inconsistency.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
How does credit-to-go differ from using only each token's own delta in HRPO?
Hierarchical Residual Policy Optimization for Generative Recommendations
Show answer and example
Answer: A single-step delta ignores how early choices constrain later ones. Credit-to-go sums residuals from the current step through the end, preserving long-term consequences. It can also accumulate estimation noise, and RRPO uses conservative updates to limit drift. The mechanism alone guarantees neither variance reduction nor an unbiased gradient of the original objective; those need a derivation for the specific case.
Illustrative example: Illustration: Choosing cameras early makes a good lens reachable later. Summing later credits lets the first token see that benefit, but an erroneous later utility can also propagate backward.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Why does HRPO decompose prefix utility into residual credits?
Hierarchical Residual Policy Optimization for Generative Recommendations
Show answer and example
Answer: Final item rewards are sparse, and giving all SID tokens the same score obscures which steps contribute. Differences in prefix utility supply the marginal credit of a newly added code, and credit-to-go accumulates later effects for earlier decisions. Smoothing depends on user clusters and estimation quality; a difference is not a directly observed causal contribution.
Illustrative example: Illustration: The first token correctly chooses outdoor goods, but the next goes to irrelevant decoration. The two can receive different feedback despite the same failed final outcome.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Core mechanism
Why ask the teacher only for zero-variance failures, not every tied group?
Show answer and example
Answer: All-fail normalization has no contrast, so teacher examples and high-entropy/gap distillation add signal. Tied successes don't need that intervention. Check outcome signs and teacher reliability.
Illustrative example: Request a verified demonstration for four wrong solutions, not four valid alternatives.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Implementation details
Does implementing a negative-group gate complete RSTG?
Show answer and example
Answer: No. Execute local distillation and teacher-success supervision with provenance/masks. Local candidate objectives do not run large teacher generation or full budget.
Illustrative example: A need_teacher flag does not execute the teacher branch.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Method comparison
How does RSTG token selection differ from imitating full teacher trajectories?
Show answer and example
Answer: Guide high entropy or large-gap positions more and add SFT on correct teacher traces. Selection avoids some irrelevant signals, but entropy/gap isn't ground-truth localization.
Illustrative example: Focus on uncertain reasoning rather than stable formatting.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Basics
Why specifically recover all-fail groups in RSTG?
Show answer and example
Answer: They lack relative reward signal and need teacher correction. All-success also has zero variance but no same failure need; consider variance and reward level.
Illustrative example: Don't call four correct replies a failure group just because their scores tie.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why cache global/local behavior and cross-attend only from the short service stream?
TransX: Scaling Transformer-based Recommendation via Behavioral and Serving Stream Crossings
Show answer and example
Answer: Behavior changes slower than requests, allowing historical encoding amortization. Global represents overall interests, local recent detail. Cache building/refresh still cost work, and candidate-dependent states are not public user caches.
Illustrative example: Reuse long history across opens and refresh local data without re-encoding years per candidate.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why combine duplicate sparse row gradients before squaring?
TransX: Scaling Transformer-based Recommendation via Behavioral and Serving Stream Crossings
Show answer and example
Answer: Squaring is nonlinear: sum logical-step gradients first. Small A100 tensors verify this rule, not the 32-H200 system or an upgraded adapter grade.
Illustrative example: Opposite gradients should cancel before the square.
Sections in the Chinese source: 补充核心公式, 补充报告效果与本地验收. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does shared multi-cutoff pooling compare with independent pooling?
TransX: Scaling Transformer-based Recommendation via Behavioral and Serving Stream Crossings
Show answer and example
Answer: Merge softmax statistics to reuse prefix work, with equivalent forward and gradients. This is compute reuse, not a changed score. Base, Retrieval, and Ranking are distinct production tests.
Illustrative example: Shared pooling should match independent computation on the same input.
Sections in the Chinese source: 补充核心公式, 补充报告效果与本地验收. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why split behavior and request streams?
TransX: Scaling Transformer-based Recommendation via Behavioral and Serving Stream Crossings
Show answer and example
Answer: Cache global/local historical states asynchronously, then let short candidate requests cross-attend. Share only candidate-independent portions.
Illustrative example: Several candidates read one user cache but retain separate matching.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Core mechanism
Why does RecHarness allocate experiments by gain and uncertainty rather than always choose the best observed architecture?
RecHarness: A Bandit-Routed Agentic Harness for Self-Evolving Recommender Systems
Show answer and example
Answer: Greedy mean selection can abandon underexplored directions too early. Confidence bonuses let uncertain arms receive experiments and winners inform later candidates. It allocates finite research budget, not globally optimal architectures, and gains require one validation protocol.
Illustrative example: Illustration: An arm tried once may merit exploration against a slightly stronger arm tried ten times; final choice still needs independent test.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Implementation details
How handle RecHarness na=0 and multi-round records without fabricated trials?
RecHarness: A Bandit-Routed Agentic Harness for Self-Evolving Recommender Systems
Show answer and example
Answer: Prioritize untested arms or define infinite exploration explicitly. Increment counts only for real completed trials with fixed validation outcomes. Persist actual state on interruption; submission is not feedback. Small proxies do not establish production research throughput.
Illustrative example: Illustration: Record training failure as a failure, not a zero validation reward or successful completed trial.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Method comparison
When is UCB preferable to equal round-robin experiments?
RecHarness: A Bandit-Routed Agentic Harness for Self-Evolving Recommender Systems
Show answer and example
Answer: It spends more on high or uncertain gains rather than repeatedly testing poor arms. Nonstationarity and unequal evaluation noise can distort estimates. Exploration does not correct unfair baselines or leakage.
Illustrative example: Illustration: An arm with five regressions can receive less budget than a promising arm, provided exploration avoids premature rejection.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Basics
Why discretize RecHarness architecture changes into finite arms?
RecHarness: A Bandit-Routed Agentic Harness for Self-Evolving Recommender Systems
Show answer and example
Answer: Explicit actions support gain and uncertainty estimates for budget allocation, with feedback guiding future trials. The bandit manages experiments, not direct recommendation. Predefined actions limit the search space.
Illustrative example: Illustration: Widening, gate replacement, and adding content are separate arms, not arbitrary novel code generated from scratch.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why does GALA use behavior rewards and gated ID fusion after image-text pretraining?
Show answer and example
Answer: Content alignment does not ensure personalized consumption relevance. Behavior rewards adapt multimodal representations, while a gate balances semantic and collaborative ID evidence per example, including cold starts. Biased rewards or a collapsed gate can fail; multimodal retrieval is not final recommendation value.
Illustrative example: Matching product images and copy do not make a product right for this user; established and new items can rely on different evidence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
Why align dimensions and item identity before applying GALA's sigmoid gate?
Show answer and example
Answer: Elementwise gates need equal-width multimodal and ID vectors for the same candidate, and their inputs must match that pair. Bounded sigmoid values cannot repair mismatches; handle missing cold-start IDs explicitly and distinguish local proxies from full training.
Illustrative example: Mixing item A's image/text vector with item B's ID can yield finite values without representing either item correctly.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
How does GALA's behavioral GRPO differ from image-text pretraining in supervision?
Show answer and example
Answer: Pretraining aligns queries, images, and text. Behavior-driven post-training compares options under actual recommendation targets rather than content similarity alone. Exposure bias and loss of semantic capability remain possible; a public proxy does not establish full policy training.
Illustrative example: Visually similar coats can perform differently for commuter users; behavioral alignment can capture that beyond correct image-text pairing.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why are GALA multimodal and ID representations complementary?
Show answer and example
Answer: Content transfers semantics and IDs retain platform behavior history. A contextual gate chooses their contributions, and behavior rewards align pretrained semantics to recommendation. Neither replaces the other universally, particularly with new items lacking feedback.
Illustrative example: A new product relies on image/text use cues, whereas a popular older one has rich click history.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Core mechanism
How does parameter memory from hidden states differ from retaining original history?
TransMem: Transforming Hidden States into Memory for Large Language Models
Show answer and example
Answer: TransMem converts sparse states from a frozen backbone to reusable gated deltas. It avoids re-reading the same original tokens but trades reuse for detail lost in selection/compression. Learned gating does not retrain the backbone per history.
Illustrative example: A compressed document can support later queries without recovering an omitted exact ID.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Implementation details
Should TransMem replace the original hidden state?
TransMem: Transforming Hidden States into Memory for Large Language Models
Show answer and example
Answer: No. h′=h+g⊙T(H_sparse) preserves the main path. Keep selection, transformation, and gated injection distinct. The mini-suite is not paper-scale training.
Illustrative example: With g=0, output is h rather than zero.
Sections in the Chinese source: 核心公式, 复现边界. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Method comparison
How does parameter memory differ from appending old text?
TransMem: Transforming Hidden States into Memory for Large Language Models
Show answer and example
Answer: T(H_sparse) transforms selected states into hidden-state updates, while text concatenation re-reads original tokens. This changes storage and fusion, and omitted detail may be lost.
Illustrative example: Keeping a fact representation does not reproduce every omitted word.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Basics
Does a frozen TransMem backbone mean nothing learns?
TransMem: Transforming Hidden States into Memory for Large Language Models
Show answer and example
Answer: No. The backbone stays fixed but evidence-conditioned self-distillation trains memory gating. Changing memory is not updating backbone weights.
Illustrative example: A new branch can learn when to inject history into the current state.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Core mechanism
Why represent tools as input-to-output hyperedges rather than retrieve by name similarity?
HyperAgent: Planning and Acting over Tool-Schema Hypergraphs for Tool-Use LLM Agents
Show answer and example
Answer: A tool can require several inputs jointly and produce multiple outputs. A hyperedge captures those requirements. The task DAG exposes missing state, then producer relations identify tools that can fill it. Name similarity does not guarantee required arguments or outputs for subsequent steps.
Illustrative example: Example: Invoicing requires an order ID and tax ID. Finding the order ID is insufficient; the hypergraph looks for a tax-ID producer rather than invoking the tool immediately.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Implementation details
When expanding a producer, how do you avoid treating its outputs as existing state?
HyperAgent: Planning and Acting over Tool-Schema Hypergraphs for Tool-Use LLM Agents
Show answer and example
Answer: The graph says a call can produce a field, not that it has. Verify actual input availability before calling and update state only from real results. Local deterministic PlanBench verifies the dependency path, not AppWorld. A complete graph or perfect fixture result cannot replace actual API responses and task verification.
Illustrative example: Example: A customer lookup schema says it returns customer_id. Perform the lookup and verify the result rather than filling an ID from the schema.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Method comparison
Which describes demand and which describes supply: the task DAG or producer support graph?
HyperAgent: Planning and Acting over Tool-Schema Hypergraphs for Tool-Use LLM Agents
Show answer and example
Answer: The task DAG describes dependencies between goal states and exposes inputs missing for later subtasks. The producer support graph uses tool schemas to identify suppliers and their own input requirements. Together they form an executable tool chain. A producer's ability to return a value does not mean it already exists.
Illustrative example: Example: The refund task DAG exposes a missing order ID. The support graph finds the order lookup tool and checks whether its required user ID can be obtained first.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Basics
Why is a name-relevant tool insufficient for a multi-step tool task?
HyperAgent: Planning and Acting over Tool-Schema Hypergraphs for Tool-Use LLM Agents
Show answer and example
Answer: A call is possible only when its inputs are present, and its outputs may be required later. HyperAgent connects task requirements to these production dependencies and fills state gaps instead of selecting the closest interface by text. Multiple valid transitions are needed for an executable workflow.
Illustrative example: Example: Finding a refund tool does not make a refund possible if it needs an order identifier and payment record. First produce those inputs.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why propagate co-engagement, quantize, then ground generated tokens?
LLM-Based Generative Retrieval for Snapchat Content Recommendation
Show answer and example
Answer: Graphs add collaborative signals, quantization encodes them in discrete paths, grounding ties codes back to content, and next-SID prediction learns transitions. Popularity bias can propagate too.
Illustrative example: Visually different co-consumed items can become neighbors, but codes still must resolve reliably to relevant content.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
What behavior can graph propagation and quantization use?
LLM-Based Generative Retrieval for Snapchat Content Recommendation
Show answer and example
Answer: Only allowed training co-engagement, without future test edges. Pin codebook and mapping versions. Public features do not reproduce Snap logs, scale, or traffic.
Illustrative example: Test clicks in the graph leak retrieval relationships.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
What does grounding SID tokens add beyond discrete-code likelihood?
LLM-Based Generative Retrieval for Snapchat Content Recommendation
Show answer and example
Answer: Alignment preserves content meaning instead of learning only code transitions. Its benefit depends on embedding and graph quality, not any arbitrary semantic code.
Illustrative example: Related videos' code representations should reflect their content relationship.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Why build a co-engagement graph before SnapLGR SIDs?
LLM-Based Generative Retrieval for Snapchat Content Recommendation
Show answer and example
Answer: Propagation adds collaborative relationships, residual quantization turns them into hierarchical paths, and generation predicts codes from history. Behavior complements content.
Illustrative example: Visually different items consumed by one audience can become connected.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Core mechanism
Why does ROCS need candidate isolation to avoid redoing user computation for every candidate?
ROCS: Request-Oriented Compute Sharing for Efficient Large-Scale Recommendation
Show answer and example
Answer: A state that has read one candidate is not a common cache. Generalized Layer Masking preserves isolation and Deep Cross Attention delays candidate interaction, so a request representation is computed once. Kernel broadcasting further reduces copies. The local in-process microbenchmark compares repeated and shared encoding, not full production throughput.
Illustrative example: One request scores a hundred items. Encode the user independently, then broadcast to each item branch. If item A contaminates user state first, reusing it for item B leaks information.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Implementation details
Does local ROCS implement production GLM masks and IKBO CUDA broadcasting?
ROCS: Request-Oriented Compute Sharing for Efficient Large-Scale Recommendation
Show answer and example
Answer: No. It covers the division between shared representation and scoring, not a specialized kernel or Meta cluster. Architectural reuse and on-device broadcasting need distinct evidence, and throughput claims require measurement.
Illustrative example: A Python loop reusing one vector does not show that a GPU kernel shares loads.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Method comparison
Are request-representation sharing and IKBO's reduced on-device copying the same optimization?
ROCS: Request-Oriented Compute Sharing for Efficient Large-Scale Recommendation
Show answer and example
Answer: No. Model information flow makes representations shareable; device broadcasting moves a valid shared representation into candidate computation efficiently. Caching a user vector can verify the division of work, not a production broadcast kernel or throughput. Each layer needs its own implementation and measurements.
Illustrative example: First ensure candidate inputs cannot contaminate the user state. In-process cache timings do not replace GPU broadcast kernel measurements.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Basics
What are the units of computation in ROCS: one request and its candidates?
ROCS: Request-Oriented Compute Sharing for Efficient Large-Scale Recommendation
Show answer and example
Answer: A request contains user-side context. Candidates are separate items scored within it. They share user input but have separate features and outputs. More candidates do not mean more independent requests. Distinguish these axes to identify reusable versus per-candidate work.
Illustrative example: One user opening a page that scores one hundred items is one request with one hundred candidates, not one hundred user requests.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why jointly train masked prediction and sequence contrastive learning in Open Web UFM rather than one CTR label?
Building a User Foundation Model for the Open Web
Show answer and example
Answer: Masking recovers local events from partial context; contrastive views teach whole-sequence consistency. Pretraining can support CTR and win-rate heads with sparse labels. Identity discontinuity and privacy affect input distribution; a local zero gain is not evidence of cross-task benefit.
Illustrative example: Illustration: A three-visit trajectory without click labels still supports masked recovery and two-view positives before fitting a small CTR head.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
Why make UFM recipe lifters auditable rather than let them change data and evaluation freely?
Building a User Foundation Model for the Open Web
Show answer and example
Answer: Define permitted changes to attribute feedback while preserving budgets and data contracts. Changing examples or metrics can create fake gains; LLM proposals cannot edit answers. Retain the zero result under the same encoder and 45 steps.
Illustrative example: Illustration: Varying mask rate is comparable; deleting hard test users changes the task rather than improving pretraining.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
What kinds of consistency do masked recovery and sequence InfoNCE enforce?
Building a User Foundation Model for the Open Web
Show answer and example
Answer: MLM predicts missing events from local context, while contrastive learning aligns whole-trajectory views. They complement each other but strong corruptions or false negatives can mislead. The local zero result does not verify transfer merely because both losses exist.
Illustrative example: Illustration: Nearby visits can recover one masked visit, and partial trajectories can retain the same intent, unless the views now represent genuinely different needs.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why can't Open Web UFM assume a stable long-term user ID and complete history?
Building a User Foundation Model for the Open Web
Show answer and example
Answer: RTB identity depends on permissions, platforms, and privacy, creating short disconnected paths. Learn transferable features from limited browsing for CTR and auction tasks. Logged-in MovieLens histories cannot directly verify open-web identity modeling.
Illustrative example: Illustration: One person can appear as several short device-specific paths. Do not silently reconstruct a universal ID.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why route HA-MoE experts by content heterogeneity and monitor domain DL-AUC?
Heterogeneous Ranking in Industrial-Scale Recommender Systems: A Case Study
Show answer and example
Answer: In mixed feeds, frequent domains can dominate shared gradients. Contextual gates combine general and specialist experts, and per-domain metrics plus utilization reveal degradation masked by averages. More experts do not prove specialization.
Illustrative example: News volume can hide sports-video losses in aggregate scores, which domain monitoring exposes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Over what axis should HA-MoE apply gate softmax, and what should LENS group by?
Heterogeneous Ranking in Industrial-Scale Recommender Systems: A Case Study
Show answer and example
Answer: Normalize per example over experts and align expert outputs before mixing. Record utilization by domain and time. Local session entropy proxies heterogeneity and measures a fixed expert set, not all open-web content.
Illustrative example: Give one news request weights over four experts; do not normalize news and video examples together.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
Why can LENS per-domain DL-AUC disagree with aggregate AUC?
Heterogeneous Ranking in Industrial-Scale Recommender Systems: A Case Study
Show answer and example
Answer: High-volume domains dominate aggregates. Slice metrics and expert usage show which types are ignored and where gains originate. Monitoring does not guarantee learning; local NDCG did not rise, and lower head share is not a quality substitute.
Illustrative example: If video is 90% of traffic, a slight video gain can mask a large news regression.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why choose HA-MoE experts by heterogeneity instead of one fixed model per content type?
Heterogeneous Ranking in Industrial-Scale Recommender Systems: A Case Study
Show answer and example
Answer: Independent models cannot share common knowledge, while one tower can favor high-volume content. Gated combinations share useful general signals while retaining specialization, which must be measured rather than assumed from expert names.
Illustrative example: Sports news can use both a freshness expert and a sports specialist.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why does Feedback Policy distill teacher relations instead of only independent item scores?
From Understanding to Action: Feedback-Grounded Policy Discovery for Generative Recommendation
Show answer and example
Answer: Knowledge also resides in candidate similarities and preferences. Relation matching transfers feedback-derived structure to a lightweight ranker without serving an LLM. Teacher biases can transfer, and smaller matrix error need not improve user feedback.
Illustrative example: Two similarly scored products can be substitutes for a request. Capturing that relation informs list composition beyond copying static scores.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Why use the same identity order for teacher and student relation matrices?
From Understanding to Action: Feedback-Grounded Policy Discovery for Generative Recommendation
Show answer and example
Answer: Every cell denotes a specific pair, so wrong ordering fits wrong relationships despite equal shapes. Match valid entries, normalization, and padding masks and use allowed evidence. The local proxy does not validate the complete real-user feedback loop.
Illustrative example: Camera in teacher row two and novel in student row two is an invalid comparison; align by item ID first.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
How do relation-matrix and per-item logit distillation emphasize different errors?
From Understanding to Action: Feedback-Grounded Policy Discovery for Generative Recommendation
Show answer and example
Answer: Logits match absolute outputs; matrices match relative layouts and can be less sensitive to global scale shifts. Structural matching can miss probability calibration. Compare under a fixed protocol rather than assuming more constraints improve quality.
Illustrative example: Adding a constant to all teacher scores leaves order intact but changes independent errors; relative distillation need not constrain absolute click probability.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Why turn Feedback Policy feedback into relational supervision rather than caching strategy text?
From Understanding to Action: Feedback-Grounded Policy Discovery for Generative Recommendation
Show answer and example
Answer: Feedback reveals similarity and substitution structures that a lightweight model can learn from a relation matrix without online LLM explanations. The matrix is a teacher proxy with possible bias and incomplete reasoning. Text storage alone does not update model behavior.
Illustrative example: Learning that two tutorials are substitutes can prevent them dominating a list; a note to diversify does not change embeddings.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why does CCFormer expand compression receptive fields for distant history while retaining recent detail?
Show answer and example
Answer: Old sequences are large, and per-event interactions can consume ranking budget. Recent behavior can signal changing intent and lose detail if merged early. CCFormer hierarchically mixes old tokens while keeping field-specific projections and cross-attention. Sequence shortening and field interaction solve different problems, so length alone cannot measure effectiveness.
Illustrative example: Merge many similar views from last month but preserve today's views of two different sizes; candidate fields can still query compressed long-term interests.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
How does hierarchical token mixing in CCFormer differ from compressing the whole sequence uniformly?
Show answer and example
Answer: Expand the old-history receptive field by layer and temporal region, keep the recent fine-grained path, and use field-specific query projections for interactions. Uniform merging loses temporal resolution allocation. For the small local core, verify connectivity and length changes rather than just runnable tensor shapes.
Illustrative example: The first layer can keep four recent actions as individual tokens and merge eight older ones. A later layer expands the old window without averaging away a just-recorded key click.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
What do CCFormer's progressive compression and fixed last-N truncation each lose?
Show answer and example
Answer: Truncation removes distant events entirely; compression retains coarse older signals at less cost but can erase rare details. Keeping recent resolution while broadening old receptive fields is not lossless. A local GRU comparison does not establish a ranking of all long-sequence methods.
Illustrative example: Compression can retain last year's photography interest and three recent camping actions but forget one camera model. Last-ten truncation may forget photography entirely.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why project CCFormer fields separately instead of treating IDs, content, and history as homogeneous tokens?
Show answer and example
Answer: Different sources and scales benefit from their own projections, followed by cross-attention, which limits dominance by a strong field. Hierarchical compression separately controls history cost. This jointly addresses feature interaction and sequence length, not simply Transformer depth.
Illustrative example: A user ID is a sparse lookup and content represents semantics. Separate projections preserve both before interaction instead of favoring frequent IDs.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Core mechanism
Why project teacher correction onto visual evidence and keep only aligned components?
Show answer and example
Answer: Matched views define a visual direction. Keep positive projection and anchor target to student; weakly regularize toward full teacher. This avoids broad stylistic imitation. Local reward axes don't reproduce visual training.
Illustrative example: Try to transfer support for a red object without importing unrelated teacher phrasing preferences.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Implementation details
What should VAD evidence-present and evidence-removed branches share?
Show answer and example
Answer: Same frozen teacher, question and student prefix; only evidence view differs. Center logp and stabilize projection. Local process proxy lacks real crops/checkpoint/benchmarks.
Illustrative example: An unrelated text model would confound model architecture with evidence ablation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Method comparison
How does one-sided projection differ from replacing all correction with the visual direction?
Show answer and example
Answer: Keep only teacher correction positively aligned to evidence. Replacement ignores whether teacher supported that change. Projection constrains attribution, not correctness.
Illustrative example: If teacher correction opposes visual contrast, don't blindly inject it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Basics
Why rebuild VAD targets around the current student?
Show answer and example
Answer: Full teacher copying imports style/language bias. Add only visual-direction corrections around student to preserve other preferences; full privileged distribution is weak regularization.
Illustrative example: Transfer image color evidence without forcing all wording changes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Core mechanism
Why use a fixed anchor and conflict-weighted context corrections?
Show answer and example
Answer: Replacing context can make targets jump. Relative corrections preserve the context-free base; geometric overlap weakens disagreement. Its normalization constant isn't arbitrary pairwise opinion distance. Local axes do not perform real extraction.
Illustrative example: For opposing suggested reasoning paths, retain the shared anchor and attenuate corrections rather than follow every new prompt.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Implementation details
Do local Flux reward axes reproduce real evolving-context extraction?
Show answer and example
Answer: No. They proxy teacher changes without video, medical or language context models. Build log-space targets relative to one anchor; don't mix differences from unrelated baselines.
Illustrative example: Two hand-built logit tensors don't implement document extraction and updates.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Method comparison
Why weaken Flux-OPD corrections when geometric teacher views conflict?
Show answer and example
Answer: Geometric normalization reflects overlap. Large conflict reduces effective correction instead of averaging incompatible suggestions forcefully. Agreement also doesn't prove correctness.
Illustrative example: Two confident opposing answers should not receive twice the trust.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Basics
Why keep a context-free anchor in Flux-OPD?
Show answer and example
Answer: Students can absorb fixed context, but replacing it can abruptly change targets. The anchor stays fixed and accepts relative corrections from evolving contexts rather than wholesale replacement.
Illustrative example: A new retrieval background can change one judgment without moving all language targets.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Core mechanism
Why distinguish reflective and exposure-bias positions rather than reward all high entropy?
Contrastive Reinforced Policy Optimization via Privileged Self-Distillation
Show answer and example
Answer: Uncertainty can be useful exploration or failure from wrong prefixes. Contrast and privileged self-distillation handle them separately. Entropy alone isn't correctness; no complete reasoning training locally.
Illustrative example: High entropy during a useful recheck and a confused continuation call for different evidence-based updates.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Implementation details
Does local CRPO identify real reflection positions?
Contrastive Reinforced Policy Optimization via Privileged Self-Distillation
Show answer and example
Answer: No. Candidate objectives test masks/diagnostics only. Record position/group provenance, keep privilege out of rollout and never label test positions with gold then claim autonomy.
Illustrative example: Predefined masks don't demonstrate finding exposure bias in real text.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Method comparison
How does CRPO contrast differ from minimizing privileged teacher KL alone?
Contrastive Reinforced Policy Optimization via Privileged Self-Distillation
Show answer and example
Answer: Contrast creates relative competition between positive/negative positions; KL directly matches a target. Classification needs reliability; negatives don't eliminate bias automatically.
Illustrative example: Which negatives appear in the denominator changes the update.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Basics
Why distinguish reflective exploration from exposure bias in CRPO?
Contrastive Reinforced Policy Optimization via Privileged Self-Distillation
Show answer and example
Answer: Both can be uncertain. Reflection can repair reasoning; exposure bias can compound prefix errors. Group contrast treats them differently instead of rewarding entropy uniformly.
Illustrative example: Rechecking a branch is not the same as drifting after a wrong equation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Core mechanism
Why combine geometric teacher/reference interpolation with return-to-go?
Show answer and example
Answer: Interpolation controls target direction/strength; return-to-go controls temporal credit. Vanilla is a specific setting, not proof arbitrary logit averages solve the same objective. Local positions aren't long sequences.
Illustrative example: How much to trust teacher and how far later errors influence early choices are separate decisions.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Implementation details
Does local β-OPSD execute long CoT with its linear-position proxy?
Show answer and example
Answer: No. Candidate actions and linear returns replace tokens/trajectory positions. Freeze teacher/reference and preserve ordering; proxy results aren't Qwen LoRA evidence.
Illustrative example: Suffix sums over a few numbers don't demonstrate equivalent credit in real generation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Method comparison
How does return-to-go differ from a current-token log gap?
Show answer and example
Answer: It accumulates discounted future gaps, assigning later consequences to current decisions. Longer horizons can add variance and depend on termination; not automatically more accurate.
Illustrative example: A single early error can cause several later deviations invisible to immediate gap.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Basics
Why view β-OPSD as a case of regularized policy optimization?
Show answer and example
Answer: Reference and privileged teacher define a geometric target; token-logit distillation replaces expensive policy optimization. β has the derivation's specific role, not arbitrary mixture weight.
Illustrative example: Same guidance with different regularization yields different effective targets.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Core mechanism
How is WIDE more fine-grained than skipping layers?
WIDE: Boosting Adaptive LLM Inference via Token-level Dynamic Width Pruning
Show answer and example
Answer: It chooses attention-head and FFN-channel groups per token within a layer. Real savings require mask rearrangement, block skipping, and device kernels. Dense zero-masking alone pays full compute.
Illustrative example: An easy token may use fewer groups than a hard token, but only skipped execution improves latency.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Implementation details
How should WIDE masks interact with kernels to save work?
WIDE: Boosting Adaptive LLM Inference via Token-level Dynamic Width Pruning
Show answer and example
Answer: Coordinate mask rearrangement, block skip, and on-device skipping. Multiplying a dense output by a mask does not remove matrix multiplication. Local PyTorch results are routing/sparsity/quality checks, not CUDA speed.
Illustrative example: A zero head output does not prove its QK product was skipped.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Method comparison
How does per-token width routing differ from pruning half the channels permanently?
WIDE: Boosting Adaptive LLM Inference via Token-level Dynamic Width Pruning
Show answer and example
Answer: Fixed pruning removes the same groups; dynamic routing selects different groups by token. Even equal average sparsity has different paths and scheduling costs.
Illustrative example: Two tokens can each use half the groups without using the same half.
Sections in the Chinese source: 背景与主要改动, 本地复现. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Basics
Why give different tokens different model widths?
WIDE: Boosting Adaptive LLM Inference via Token-level Dynamic Width Pruning
Show answer and example
Answer: Static pruning uses one subnetwork for all inputs. WIDE routes attention head groups and FFN channels separately to allocate token-specific compute within layers.
Illustrative example: Simple positions can use few groups while complex evidence gets more.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Core mechanism
Why score retrieval in the final-layer value space in ReToken?
ReToken: One Token to Improve Vision–Language Models for Visual Retrieval
Show answer and example
Answer: Prefilled vision KV already contains image representations. Similarities in the same value space can reuse cache without rerunning an external retriever and re-encoding images. A learned retrieval token and projection adapt relevance; a frozen VLM does not eliminate supervision or guarantee generalization.
Illustrative example: With 10 prefetched images, score their average value vectors against the new query's retrieval token and reuse selected images.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Implementation details
What is missing between ReToken's local text cache experiment and the visual training?
ReToken: One Token to Improve Vision–Language Models for Visual Retrieval
Show answer and example
Answer: WikiText-2 trains a single retrieval target, value scoring, and sparse selection, not the frozen large VLM, MIRAGE vision-language supervision, or long video. Short prefixes cannot fill top-K and affect retention. Report worse PPL as a text mechanism result, not a visual gain.
Illustrative example: With three history tokens and top-K ten, at most three can be retained; average retention across lengths is not a fixed compression ratio.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Method comparison
How does ReToken per-image BCE differ from softmax with one positive candidate?
ReToken: One Token to Improve Vision–Language Models for Visual Retrieval
Show answer and example
Answer: BCE scores each relevance label independently, weighting classes without forcing sigmoid scores to sum to one. Single-positive softmax makes candidates share probability mass. Do not silently discard valid additional positives while claiming the same supervision.
Illustrative example: A question can require two relevant images. Both can be positive rather than relabeling one negative to force one winner.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Basics
Is ReToken's learned RET token a fixed ID for one image?
ReToken: One Token to Improve Vision–Language Models for Visual Retrieval
Show answer and example
Answer: No. It is a shared learned input marker whose contextual VLM state is projected and matched to each image's average value. Shared parameters do not imply identical query states or selected images, and the token does not encode an entire catalog.
Illustrative example: Two questions with the same RET marker can yield different relevance scores for the same image pool.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
Why can learning next observations from reused rollouts add training signal without adding deployment inference steps?
Show answer and example
Answer: Each transition already has state, action, and next observation. An auxiliary prediction target on the shared backbone uses them without new expert or policy rollouts. Deployment keeps the original policy without explicitly calling a predictor. Controlled comparisons must establish task gains; predicting feedback does not imply choosing good actions.
Illustrative example: An ls command already returns a directory listing. Train transition prediction from it, but deploy the policy to choose commands without first imagining a listing at each step.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
Why check that s_t, a_t, and the next observation belong together in TAPO targets?
Show answer and example
Answer: Misalignment or cross-episode concatenation supervises the wrong action result. Targets must come from legal feedback in the same rollout, without exposing s_{t+1} in prediction input. High mini-suite accuracy is not real tool modeling capability.
Illustrative example: Step two should predict the third observation it caused, not the first observation in the next task.
Sections in the Chinese source: 核心公式, 复现边界. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
Is TAPO's training-time transition prediction the same as an extra world-model call at deployment?
Show answer and example
Answer: No. Its auxiliary objective trains a shared backbone without requiring extra prediction at deployment. Explicit world-model planning adds online calls. More training supervision need not add inference cost.
Illustrative example: The model can learn next-observation prediction and still generate only the original action before using a tool.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why does TAPO reuse observed environment transitions to add training signal?
Show answer and example
Answer: A rollout contains each post-action observation as well as terminal reward. Predicting s_{t+1} from (s_t,a_t) teaches the shared backbone action consequences without additional expert demonstrations.
Illustrative example: Use actual tool responses to teach the model what feedback its actions produce.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Core mechanism
Why can ordinary accuracy conceal a reward model that predicts almost everything successful?
Show answer and example
Answer: With many successes, approving every trace can score well while missing actual failures. Report success recall, failure recall, and their mean. Cross-system and hard-subset conclusions need real labels; interface diagnostics do not measure open judge capability.
Illustrative example: At 90% successes, predicting all successes gives 90% accuracy and zero failure recall, so balanced accuracy is only 50%.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Implementation details
Did the local OSReward path train an open 9B or 35B vision judge?
Show answer and example
Answer: No. It did not download official screenshot trajectories either; it only implements the evaluation contract. Join predictions with human labels on the grading side. Explicitly report empty class denominators rather than filling fake balanced accuracy.
Illustrative example: A batch with no failures cannot estimate failure recall.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Method comparison
How does a judge reading the last five states plus text history differ from using only the final screenshot?
Show answer and example
Answer: A final image can omit process and implicit requirements, while history supplies evidence but also noise. An arbitrary summary changes the observation protocol and is not directly comparable with the paper's scores.
Illustrative example: An open window at the end does not show that prior file edits were saved.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Basics
Does OS-Shepherd operate the computer or judge trajectory success?
Show answer and example
Answer: It judges complete computer-operation traces. The evaluated actor clicks and types. OSReward provides human-verified tasks and trajectories for testing reward models. Do not confuse recognizing success with performing the task.
Illustrative example: The actor attempts to save a file and the judge checks the trace. A correct verdict does not mean the judge saved it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Core mechanism
Why adjust topology edges, ordering, and visibility rather than merely add agents?
MANTA: Multi-Agent Network Topology Adaptation for Self-Evolving Multi-Agent Systems
Show answer and example
Answer: Failure can come from missing information, duplicate roles, or absent verification. More agents do not fix these. Trajectories guide bounded changes to relationships and information flow. Budget projection limits communication and execution, and only task feedback confirms gains.
Illustrative example: Example: If reviewers never see current code, fix message visibility rather than add three reviewers of the same stale material.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Implementation details
What does Π_budget do in topology updates, and what does local evidence establish?
MANTA: Multi-Agent Network Topology Adaptation for Self-Evolving Multi-Agent Systems
Show answer and example
Answer: Project proposed roles, edges, and order changes into allowed resources before execution rather than running an oversized team and accounting for only part of it. The deterministic suite checks state and scheduling without real multi-agent models or environments. Topology change counts are not five-benchmark generalization.
Illustrative example: Example: If a proposal asks for ten roles but only four fit, construct an executable bounded graph before running, not after omitting six costs.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Method comparison
What is the trade-off between online topology adaptation and picking a team once at the start?
MANTA: Multi-Agent Network Topology Adaptation for Self-Evolving Multi-Agent Systems
Show answer and example
Answer: An initial team uses task priors. Online changes respond to observed bottlenecks, failures, and duplication but can add instability and cost. Project changes onto budgets and permission boundaries rather than add unlimited agents or expose private histories.
Illustrative example: Example: Merge duplicate source-checking roles or alter edges, but never let evaluator references reach executors as part of the change.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Basics
How can poor communication waste opportunities in a multi-agent team?
MANTA: Multi-Agent Network Topology Adaptation for Self-Evolving Multi-Agent Systems
Show answer and example
Answer: More roles do not help if results reach the wrong people, dependencies miss key messages, or ordering is wrong. MANTA adapts communication from execution rather than fixing a roster. Topology changes still cost coordination; evaluate visibility, results, and resources.
Illustrative example: Example: A verifier spots a constraint error but tells an unrelated role. The executor continues on a stale plan; adjust information flow rather than add another agent.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
Why generate DO/AVOID guidance with a stop-gradient snapshot and make it visible only during training?
Group-Reflective Self-Distillation
Show answer and example
Answer: Guidance comes from verified same-task successes and failures. A frozen snapshot prevents the guidance path from drifting with current gradients. It modulates turn credit without leaking hindsight group outcomes to evaluation policies. Positive multipliers preserve outcome-advantage direction rather than making reflection an answer or reward oracle.
Illustrative example: If successes check dependencies and failures edit directly, derive training guidance from that contrast. A new test task cannot receive reflections from its own future success and failure group.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
Why check GRSD's positive modulation m_t separately from outcome advantage?
Group-Reflective Self-Distillation
Show answer and example
Answer: m_t>0 adjusts magnitude while preserving the outcome advantage's sign; a negative value reverses the update. A stop-gradient snapshot provides training-only guidance, which must not remain in evaluation inputs.
Illustrative example: An action that should become less likely must still become less likely after multiplication by positive m.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
How does GRSD's success/failure group comparison differ from copying one successful trajectory?
Group-Reflective Self-Distillation
Show answer and example
Answer: A success can contain accidental or unnecessary steps. Group comparison uses failures to identify what to do or avoid. It is training guidance, not an oracle for independent testing.
Illustrative example: An irrelevant tool call in a successful trace is not automatically worth learning.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why does GRSD reflect on verified success and failure trajectories for the same task?
Group-Reflective Self-Distillation
Show answer and example
Answer: Terminal reward mixes useful behavior, repeated mistakes, and chance. Contrasting reflections within a task yields specific DO/AVOID guidance to modulate turn credit.
Illustrative example: Both trajectories retrieve information, but only one checks a key condition. The contrast helps isolate useful behavior.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Core mechanism
Why keep predictions verifiable after the cutoff with the post-submission evaluator?
FinanceHarness: Autonomous Financial Deep Research Framework
Show answer and example
Answer: Research may use only evidence available by cutoff. Earlier access to future materials turns prediction into repetition. Thesis and rubric also remain with the grader; normalize per item before averaging. Leakage invalidates the run rather than justifying dropping or zeroing a row.
Illustrative example: Later earnings can verify a forecast after submission, but must not be backfilled into the earlier evidence pack.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Implementation details
Can a finance-protocol row with time leakage be set to zero and included in the mean?
FinanceHarness: Autonomous Financial Deep Research Framework
Show answer and example
Answer: No. Invalidate the run rather than silently dropping or zeroing the row. publication_date must be no later than cutoff, and thesis, rubric, and future evidence stay away from the agent. This is a point-in-time evaluation protocol, not investment advice.
Illustrative example: A tool finding an article published after the cutoff is not merely one wrong answer.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Method comparison
How does equal weighting by item differ from pooling all rubrics?
FinanceHarness: Autonomous Financial Deep Research Framework
Show answer and example
Answer: Items have different counts. Pooling gives detailed items more weight; normalize per item first and then average. Do not replace the five-point rubric with an arbitrary continuous scale.
Illustrative example: An item with ten criteria should not automatically weigh five times one with two.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Basics
Is a FinanceGym rubric score of 3 a 3% success rate?
FinanceHarness: Autonomous Financial Deep Research Framework
Show answer and example
Answer: No. A rubric uses five grades from 0 through 4, and 3 is one level. Normalize within each item by its four-point maximum before averaging items. Interpretation still depends on the rubric. Raw levels, normalized item scores, and overall Outcome are distinct.
Illustrative example: For two rubrics scored 3 and 1, the item score is (3+1)/(4×2)=0.5, not 4% or a two-item hit rate.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why do PSG pair tokens reduce steps but need internal order?
PSG: Pair-Space Generation for Efficient Generative Reranking
Show answer and example
Answer: Each token jointly models two positions and can be losslessly expanded, halving autoregressive steps for even lengths. Pair order affects display and future conditions, while cross-pair duplicates can invalidate the slate. Some complexity moves to the larger pairing/scoring space.
Illustrative example: Illustration: Six items take three ordered pairs. Coffee machine then beans differs from beans then machine; another pair containing the machine needs duplicate handling.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Why deduplicate across PSG pairs instead of only within each pair?
PSG: Pair-Space Generation for Efficient Generative Reranking
Show answer and example
Answer: Local uniqueness does not prevent repeats across a slate. Check new pairs against the selected prefix and preserve internal order when expanding. Define odd lengths, leftovers, and empty legal sets instead of repeating items to fill slots.
Illustrative example: Illustration: Camera–lens and camera–bag are each unique internally but repeat the camera after expansion.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
What dependency can PSG pair scoring capture beyond parallel individual item heads?
PSG: Pair-Space Generation for Efficient Generative Reranking
Show answer and example
Answer: Joint scoring evaluates two items and their order as one action, modeling complementarity. Parallel independent heads may not. The pair action space can be large, and the proxy regressed, so half-length alone is not lossless speedup.
Illustrative example: Illustration: Camera plus lens can be better than two redundant high-scoring cameras, but all possible pairs outnumber individual items.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Does lossless PSG expansion mean pair and item models have the same probabilities?
PSG: Pair-Space Generation for Efficient Generative Reranking
Show answer and example
Answer: No. The mapping recovers both items and their order for a given pair; the scorer and search distribution can still change. Fewer steps do not imply equal training error, coverage, or quality.
Illustrative example: Illustration: Packing A B C D as AB and CD recovers it exactly, but the new model can prefer other pairs.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Core mechanism
How does OneShot train discrete hierarchical addresses on ranking rather than embedding-distance clusters?
OneShot: Index-in-Ranking with Neural Scoring for Large-Scale Retrieval
Show answer and example
Answer: Forward passes take discrete paths; straight-through gradients approximate ranking-loss feedback to assignment parameters. Global balancing avoids crowded paths, and neural scoring ranks retrieved candidates. Joint learning aligns objectives, but approximate gradients and balanced load do not guarantee relevance.
Illustrative example: Illustration: Similar text with different audiences can be assigned to different paths by behavior; balancing can keep popular items from overloading one bucket.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Implementation details
Why check OneShot's straight-through forward and backward behavior separately?
OneShot: Index-in-Ranking with Neural Scoring for Large-Scale Retrieval
Show answer and example
Answer: Forward must choose a real discrete one-hot route; backward approximates with softmax gradients. Fully soft assignment is not discrete serving, while hard argmax may block gradients. Align path, item map, and scorer versions to verify ranking trains addresses.
Illustrative example: Illustration: Routing weights receive gradients, but one forward still chooses one prescribed path, not a mix of all buckets.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Method comparison
Why do OneShot path balancing and popularity penalties affect different objects?
OneShot: Index-in-Ranking with Neural Scoring for Large-Scale Retrieval
Show answer and example
Answer: Path balancing manages bucket load and resources. Popularity penalties alter recommendation preference. Balanced buckets may spread popular items without reducing their exposure, so measure index use and business distribution separately.
Illustrative example: Illustration: Ten popular items can fill ten distinct buckets and still all be recommended.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Basics
Why make the index path part of OneShot rather than cluster after training?
OneShot: Index-in-Ranking with Neural Scoring for Large-Scale Retrieval
Show answer and example
Answer: Post-training clustering follows existing geometry and can disagree with ranking. Joint training uses behavior loss to shape buckets, then a strong scorer distinguishes retrieved items. Discrete routing and balancing can still reduce coverage and may not help at every scale.
Illustrative example: Illustration: Ranking can separate items with similar text but distinct buyers. A fixed bucket can hide the right item from even a good scorer.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why does DIRECTOR combine Sinkhorn with global matching rather than independently choosing each position's top item?
DIRECTOR: Dynamic Index-based Recommendation with Transport-Optimized Retrieval
Show answer and example
Answer: Independent predictions can select the same item. Sinkhorn constructs a soft transport plan constrained by row and column marginals; global hard matching then creates a conflict-free list. Soft probabilities alone are not a displayable permutation.
Illustrative example: If all three positions favor A, matching assigns it where most useful and places B and C elsewhere.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
How should DIRECTOR align score matrices and final permutations with candidate IDs?
DIRECTOR: Dynamic Index-based Recommendation with Transport-Optimized Retrieval
Show answer and example
Answer: Rows are positions and columns are candidate identities. Maintain column identity through transport and matching, and retrieve by matched global indexes rather than per-row rank. Explicitly handle duplicate IDs or too few candidates. Local movie proxies do not establish production valid views.
Illustrative example: If column two was movie B, indexing a reshuffled candidate table after matching can display the wrong movie.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
Why not decode DIRECTOR's Sinkhorn matrix with independent row argmaxes?
DIRECTOR: Dynamic Index-based Recommendation with Transport-Optimized Retrieval
Show answer and example
Answer: Continuous soft constraints do not guarantee that independent maxima choose distinct columns. Discrete global matching trades per-position scores for total feasible assignment quality. Temperature and iteration affect approximation, not rowwise legality.
Illustrative example: If both rows favor A at 0.6 over B at 0.4, independent choices conflict; hard matching must give B to one row.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why does DIRECTOR's dynamic index distribution cover all display positions jointly?
DIRECTOR: Dynamic Index-based Recommendation with Transport-Optimized Retrieval
Show answer and example
Answer: Positions draw from competing candidate sets. Joint transport connects position supply and item allocation, and hard matching creates a feasible slate. Parallel generation reduces sequential dependencies, but still requires relevance and legality checks.
Illustrative example: Five positions can all favor the same movie; joint allocation restricts it to its allowed count and assigns other options elsewhere.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why can ClockRoPE give higher relevance to a one-week gap than to a two-day gap?
ClockRoPE: Random Fourier Rotations for Temporal Routine Modeling
Show answer and example
Answer: Behavior may recur periodically rather than decay monotonically. ClockRoPE samples rotary frequencies from the desired periodic kernel's spectrum so query-key dot products strengthen again at matching times of day or week. Ordinary RoPE can share channels to retain recency. Periodicity is a bias, not a universal user habit.
Illustrative example: Last Monday morning commute news can be more relevant to this Monday than Saturday-night entertainment.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why must ClockRoPE's day and week periods match the timestamp unit?
ClockRoPE: Random Fourier Rotations for Temporal Routine Modeling
Show answer and example
Answer: The local kernel uses periods 24 and 168 when gaps are in hours. Raw seconds shift phase, and sampled frequencies and mixed channels must respect rotary dimension pairing. Test re-enhancement one period apart, not just tensor shapes.
Illustrative example: Convert seven days to 168 hours. Feeding the raw second count interprets the same interval as many incorrect periods.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does ClockRoPE differ from adding a weekday feature?
ClockRoPE: Random Fourier Rotations for Temporal Routine Modeling
Show answer and example
Answer: A weekday feature is an absolute time label. ClockRoPE changes pairwise attention directly through relative query-key rotations and a periodic similarity kernel. They may overlap or complement one another; the mechanism alone does not guarantee better results.
Illustrative example: A Friday label says what day it is; rotation directly relates this Friday to last Friday in attention.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why use a periodic kernel in ClockRoPE rather than assuming relevance always decays with distance?
ClockRoPE: Random Fourier Rotations for Temporal Routine Modeling
Show answer and example
Answer: Habits may repeat by hour or weekday. The periodic kernel restores similarity at specific offsets through its rotary frequencies. It is a prior, and proxies without real timestamps are not evidence of periodic behavior.
Illustrative example: A Friday film habit makes a seven-day gap informative; for an irregular viewer the same periodic bonus can add noise.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Core mechanism
Why not replace ASARL Critic and GenAgent with one LLM labeling pass?
ASARL: Autonomous Social-Aware Relevance Learning for QQ Search
Show answer and example
Answer: A one-pass labeler can inherit bias and miss long-tail intent. ReasonAgent proposes, Critic finds repeated gaps, GenAgent targets them, and behavior feedback plus distillation adapts serving. Generated rationales are not automatically truth.
Illustrative example: Diagnose campus-slang mistakes, then add matching counterexamples rather than more unrelated labels.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Implementation details
Does ASARL KL distill reasons or label distributions?
ASARL: Autonomous Social-Aware Relevance Learning for QQ Search
Show answer and example
Answer: The online student minimizes teacher/student label-distribution KL. Reasons support data and teacher training, not online explanations. Genre proxy classification improved without ranking transfer.
Illustrative example: Learn three relevance probabilities rather than generate long reasoning per request.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Method comparison
How does PGO differ from synthetic three-level supervision?
ASARL: Autonomous Social-Aware Relevance Learning for QQ Search
Show answer and example
Answer: PGO uses interaction-validated reasoning/label preference pairs, not only generated and reviewed labels. Behavior can remain biased; lower classification loss is not retrieval evidence.
Illustrative example: Real users preferring camera-model groups can correct generic photography labels.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Basics
Why learn three social relevance levels before distilling an online student?
ASARL: Autonomous Social-Aware Relevance Learning for QQ Search
Show answer and example
Answer: Colloquial, long-tail titles need contextual teacher supervision. Transfer fine-grained label distributions to a small model; validate against real search needs.
Illustrative example: “Weekend group” can mean gaming or hiking; teacher reasons in context and student predicts strong, weak, or irrelevant.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Core mechanism
Why separate SERPO archive, rubric and actor roles?
SERPO: Self-Evolving Rubric Policy Optimization for Open-Ended Test-Time Reinforcement Learning
Show answer and example
Answer: The archive stores comparisons, rubric defines current standards, and actor generates. Feedback can adapt or amplify bias. New archive or rubric doesn't itself train or improve the model; use independent holdout.
Illustrative example: A short-only archive, brevity rubric and short actor may agree while neglecting facts.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Implementation details
How should SERPO track its three evolving states?
SERPO: Self-Evolving Rubric Policy Optimization for Open-Ended Test-Time Reinforcement Learning
Show answer and example
Answer: Record versions and ordering of generation, archive, rubric and actor updates. Don't retrospectively score with later rubrics. Local candidates don't implement full generation/judging.
Illustrative example: Log the actual rubric used for each batch so training supervision is auditable.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Method comparison
How does SERPO differ from repeated RL under a fixed rubric?
SERPO: Self-Evolving Rubric Policy Optimization for Open-Ended Test-Time Reinforcement Learning
Show answer and example
Answer: It updates query-specific rubrics and archives with the actor. Fixed criteria can lose discrimination, but coevolution risks drift and self-reinforcement. Self-scores aren't proof.
Illustrative example: New criteria can distinguish harder negatives after the actor learns initial rules.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Basics
Why keep a Good–Normal–Bad archive in SERPO?
SERPO: Self-Evolving Rubric Policy Optimization for Open-Ended Test-Time Reinforcement Learning
Show answer and example
Answer: Different qualities inform prompt-specific rubrics and policy learning. Actor and criteria evolve; storing only highest scores loses contrasts. Classification depends on judge quality.
Illustrative example: Medium answers expose gaps and bad answers show patterns to avoid.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
Why downweight common replies and adjust individual-token variance separately?
Show answer and example
Answer: Response weights correct expected repetition across the group; token variance measures local plasticity. Neither replaces the other. Coverage isn't arbitrary randomness; candidate probabilities do not show long-text coverage.
Illustrative example: Downweighting repeated template responses can still leave nearly deterministic final tokens dominating.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
Why preserve old versus current probability in ReCo's Bernoulli variance ratio?
Show answer and example
Answer: Compare current p(1−p) to old variance for changes in decision saturation. Protect near-zero denominators, not use current on both sides and get one. Candidate tests aren't text pass@64.
Illustrative example: Check numerical stability when old probability is near a boundary instead of accepting an enormous anomalous signal.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
Is deduplication equivalent to ReCo expected-count weighting?
Show answer and example
Answer: Hard dedup drops repeated samples; expected-count weights vary continuously with policy likelihood. A separate variance ratio adjusts tokens. The objectives differ.
Illustrative example: Two responses may appear only once each while one has much higher underlying probability and different weight.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
Why does ReCo address both duplicate replies and saturated tokens?
Show answer and example
Answer: Common responses dominate sample weight and already-certain tokens can be reinforced repeatedly. Expected response counts and token variance ratios target different redundancies to retain exploration.
Illustrative example: Repeated solutions and their nearly certain format tokens can both dominate gradients.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why reward skill-document curation from later tasks rather than current task success?
Show answer and example
Answer: Current success mostly reflects solving, not whether the resulting skill helps later users or the same agent. Related tasks form a sequence, and discounted downstream returns supervise curation toward transferable experience. Apply the two phase advantages to their own generated tokens rather than broadcasting credit.
Illustrative example: Example: A lucky correct answer can still yield a reusable testing procedure. Its curation value is visible only if it helps later tasks.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
Do local document-update and downstream-credit counts mean Qwen3 was trained with RL?
Show answer and example
Answer: No. Controlled PlanBench tests state evolution, skill transfer, and phase-specific credit without Qwen3 training or the paper's three environments. Writes show the action happened, not parameter learning. Perfect mini-suite scores are not comparable with paper Pass@k.
Illustrative example: Example: Deterministic rules can write 120 documents. Without real gradient updates, that is not the paper's shared-policy training.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
How does a connected curriculum transfer experience differently from training episodes independently?
Show answer and example
Answer: A skill document curated from one task becomes input to the next, so transferable experience can be used and evaluated. Independent episodes normally do not expose this link. Related tasks can overfit a curriculum; distinguish downstream document benefit from current solving and do not infer generalization from transfer counts.
Illustrative example: Example: A simple table cleanup produces checks for a constrained variant next. Random unrelated tasks may never use those checks.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Why does one policy alternate solving the current task and curating a skill document?
Show answer and example
Answer: Solving produces experience; curation passes transferable material directly into the next task, making cross-task learning part of behavior. Related tasks progress from easy to hard, so documents must help later rather than simply explain present success. This is not independent episode training and not a mandate to keep all experiences.
Illustrative example: Example: After a simple file search, record location and verification methods for a harder modification task instead of copying the old answer.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Core mechanism
Why derive stop labels from the payoff gap between the current prefix and best later prefix?
Show answer and example
Answer: Relevance ranking does not say how many exposed tools are sufficient. Early stop can miss capability and late stop costs resources. Virtual payoff subtracts cost from sufficiency; the difference sets stop or continue and its magnitude gives regret. This happens before execution, so virtual payoff is not tool success.
Illustrative example: Example: Two tools already suffice, while a related but expensive third reduces payoff. Stop labels should reflect that cost rather than rank alone.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Implementation details
Can local required-tools sufficiency be a general deployment scorer?
Show answer and example
Answer: No. Required-tools lists in the auditable mini-suite diagnose prefix choices rather than act as a learned semantic scorer. Do not reveal hidden requirements to an evaluated policy. Lower exposure is not live τ-bench token or success improvement.
Illustrative example: Example: Knowing the third tool is required is useful for offline payoff, but a runtime router reading that list bypasses the learned sufficiency problem.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Method comparison
How is regret weighting different from ordinary stop/continue labels?
Show answer and example
Answer: Error costs can differ. Gap sign supplies the class and magnitude represents lost payoff. Equal weights treat near ties like missed essential tools. A best later prefix is a comparison estimate that depends on sufficiency scoring, not hidden deployment truth.
Illustrative example: Example: Exposing one cheap extra tool has little cost. Stopping before an essential refund interface is a large error that deserves greater weight.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Basics
Why decide how many tools to expose after ranking them?
Show answer and example
Answer: Rank order does not identify a sufficient set. Too many tools add cost and distraction, while too few miss interfaces. CAM-DF compares virtual prefixes and heterogeneous costs before execution. It controls exposure, not dynamic stopping after actual calls.
Illustrative example: Example: If the first two tools only query and the third modifies, a modification task may need the third regardless of the earlier scores.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Core mechanism
Why learn final conversion and delay CDF separately?
TWICE: Two-Clock, Two-Window Learning for Long-Horizon Conversion Prediction in Online Advertising
Show answer and example
Answer: An unobserved conversion may be delayed. Event probability times delay CDF gives observable status. Two clocks/windows address maturity bias, subject to a suitable delay model. Public timestamps are not ad results.
Illustrative example: Five minutes without purchase is not equivalent to a week without purchase.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Implementation details
What monotonic boundary must the delay CDF satisfy?
TWICE: Two-Clock, Two-Window Learning for Long-Horizon Conversion Prediction in Online Advertising
Show answer and example
Answer: Stay in [0,1] and never decrease with time. Keep it separate from final CVR. Public proxies lack actual ad delay, aggregates, and revenue.
Illustrative example: A two-day CDF cannot fall below the one-day value.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Method comparison
What do click-clock status and conversion-clock delay measure?
TWICE: Two-Clock, Two-Window Learning for Long-Horizon Conversion Prediction in Online Advertising
Show answer and example
Answer: One measures maturity since click, the other the conversion delay distribution. Joint windows correct observation bias; one clock can mix event likelihood with timing.
Illustrative example: Equal waits can differ in label maturity under different sampling windows.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Basics
Why distinguish not-yet-converted from never-converting?
TWICE: Two-Clock, Two-Window Learning for Long-Horizon Conversion Prediction in Online Advertising
Show answer and example
Answer: A recent click can convert later. Final CVR times delay CDF describes current observation without treating immature negatives as permanent.
Illustrative example: Today's click can become tomorrow's order.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why use a seven-day plan across episodes and inject it through per-step gates?
Show answer and example
Answer: Local rewards can miss multi-day budgets. Sliding plans carry long goals; gates let immediate markets override guidance when appropriate. Masked future supervision is training-only, and a small zero-change result does not validate bidding gains.
Illustrative example: Save budget during a crowded day but act on a valuable current opportunity using only available information.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Does local SWAG use real bids, budgets, and GMV?
Show answer and example
Answer: No. It ablates structure, not a production Decision Transformer. Maintain causal masks and timestamps, without filling from future outcomes.
Illustrative example: A movie action score is not an auction price.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
How does masked future planning in training differ from deployment future access?
Show answer and example
Answer: Training can construct masked plan signals under protocol. Serving sees only then-available state and estimates, not realized later GMV or spend.
Illustrative example: Tomorrow's spend can supervise offline training but cannot be an input to today's bid.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Are SWAG-Bid episode boundaries and seven-day windows the same?
Show answer and example
Answer: No. Episodes bound local interactions; the sliding window carries longer goals/state across episodes, not seven fixed actions. Do not reset all long-term context at every boundary.
Illustrative example: A new bidding day does not restart a seven-day objective.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why does exponential reward-guided decoding preserve a prior while depending on KL temperature?
Reward Guided Decoding for Generative Recommendation
Show answer and example
Answer: The tilted distribution multiplies prior probability by exponentiated reward. Smaller temperature follows reward more aggressively; larger approaches the prior. Search and normalization affect approximation, and reward errors can hurt even without training.
Illustrative example: Illustration: Moderate novelty can elevate relevant niche movies; too low a temperature can select irrelevant novelties.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Why implement reward tilting with logsumexp and test temperature limits?
Reward Guided Decoding for Generative Recommendation
Show answer and example
Answer: Raw exponentials overflow. Normalize in log space; large β should approach the prior, small β emphasize reward, and illegal entries stay masked. Separate truncated approximation from exact normalization. Local novelty reward reduced NDCG, so higher reward did not establish better ranking.
Illustrative example: Illustration: exp(1000) can overflow; stable normalization fixes numerics, not irrelevant novelty at low temperature.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
When does exponential tilting correspond to adding logits, and when does it not?
Reward Guided Decoding for Generative Recommendation
Show answer and example
Answer: On identical support, log Q equals log P0+R/β up to a shared normalizer. Adding reward to raw P0 instead changes the distribution. Candidate truncation also changes support and normalization and is not the exact closed form.
Illustrative example: Illustration: Adding one to probabilities 0.9 and 0.1 has a different effect from adding it in log space.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Can reward-guided decoding give positive mass to a sequence with P0=0?
Reward Guided Decoding for Generative Recommendation
Show answer and example
Answer: Not under the closed form. Multiplying strict zero by an exponential stays zero, and normalization does not create support. Smoothing, candidate expansion, or generator changes are separate steps.
Illustrative example: Illustration: A movie absent from all legal candidates cannot be recovered by assigning a high novelty reward.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why reward target minus non-target affinity in Reco-Reward rather than general appeal?
RecoReward: Recommender-Guided Multimodal Description Generation for Recommendation
Show answer and example
Answer: Target-only scores can favor generic popular copy. Subtracting non-target affinity encourages audience-distinguishing features. A frozen two-tower model provides a stable reward, and serving consumes descriptions without real-time user profiles. Proxy gains are not verified satisfaction.
Illustrative example: Illustration: Specify acidity and brewing parameters for pour-over fans rather than write 'everyone loves it.'
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
Why freeze the two towers and track non-target provenance for RAS?
RecoReward: Recommender-Guided Multimodal Description Generation for Recommendation
Show answer and example
Answer: A drifting reward model can raise scores without better descriptions, and negative-population choice changes the subtraction. Isolate generation/selection and evaluation; test users cannot supervise training. The history-centroid proxy is not the production tower.
Illustrative example: Illustration: A weaker description can score higher if the evaluator drifts; a fixed evaluator still needs independent validation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
Why is higher frozen Reco-Reward not equivalent to users liking copy more?
RecoReward: Recommender-Guided Multimodal Description Generation for Recommendation
Show answer and example
Answer: A fixed two-tower RAS estimates affinity difference and can be exploited. Freezing stabilizes the yardstick, not its correctness. Separately verify reward, fidelity, and independent behavior.
Illustrative example: Illustration: Stuffing audience keywords may raise model affinity while making text unreadable or misleading.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why optimize content descriptions rather than generate user profiles online in Reco-Reward?
RecoReward: Recommender-Guided Multimodal Description Generation for Recommendation
Show answer and example
Answer: A frozen recommender rewards audience fit while serving reads the optimized description without generating a profile. Training uses audience signals without exposing profiles online. Descriptions must remain factual despite reward pressure.
Illustrative example: Illustration: Mention real pour-over attributes; do not invent them to increase affinity.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Core mechanism
Why not count a relayed trace as purely student on-policy?
Show answer and example
Answer: Teacher generates intervened actions and changes later student conditioning. It is a mixture. Budget keeps help local and provenance tracks writers. Include teacher cost and don't claim independent success.
Illustrative example: A student finishing after two teacher steps need not succeed unaided.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Implementation details
How does the Relay state machine prevent budget overrun?
Show answer and example
Answer: Track cumulative handoff lengths, enforce total and record takeover/return/termination. Local teacher is cached candidate distribution, not real Qwen or eight-GPU training.
Illustrative example: Once budget is spent, do not start a second handoff.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Method comparison
How do relayed traces differ from native student rollouts?
Show answer and example
Answer: Teacher intervention changes action sources and states; log takeover interval and budget. Teacher-assisted success is not autonomous student success.
Illustrative example: A teacher bypassing a hard step requires separate no-teacher evaluation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Basics
Why let Relay-OPD teacher briefly take over then return control?
Show answer and example
Answer: An invalid prefix can trap future learning. Short, budgeted intervention returns to a learnable region while student still acts after handoff.
Illustrative example: Teacher repairs a few critical steps, then student finishes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Core mechanism
Why replay the same response instead of generating separately under each rubric view?
Show answer and example
Answer: Fixed tokens/prefixes isolate conditioning better; regenerating also changes trajectories. Use the difference to allocate original advantage, not pass scoring text into deployment. Counterfactual contexts can still bias the estimate.
Illustrative example: A citation token's likelihood can change with a citation requirement; regenerating the whole paragraph adds other changes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Implementation details
Why align both CoRT scoring passes to the same token and prefix?
Show answer and example
Answer: Different generated text confounds the criteria intervention. Normalized weights scale fixed advantage; original student sampling must not see the privileged rubric. Candidate features aren't token replay.
Illustrative example: Scoring sentence A with criteria and B without doesn't isolate their effect.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Method comparison
How does CoRT counterfactual replay differ from attention weighting?
Show answer and example
Answer: Replay changes evaluation context and compares probabilities; attention measures one forward's information weights. Neither directly proves causality; attention isn't the specified contrast.
Illustrative example: A high-attention token may show unchanged likelihood when criteria are removed.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Basics
Why replay one reply with and without criteria in CoRT?
Show answer and example
Answer: Keep response fixed so likelihood differences link rubric context to token positions, redistributing response advantage rather than adding another global score.
Illustrative example: A format requirement can affect bracket tokens and their credit.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
Why calibrate timing before rewarding speed?
Reinforcement Learning for Code Optimization
Show answer and example
Answer: Noise from workers and small tests can teach exploitation rather than optimization. Independent timing, correctness and performance tests define valid rewards, with sparse/stale handling afterward. Syntax/fixtures don't prove faster production.
Illustrative example: Repeat timing under matched conditions instead of rewarding a run that happened on an idle host.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
Can the local code-optimization component replace a timing service?
Reinforcement Learning for Code Optimization
Show answer and example
Answer: No. Real rewards need correctness and optimization tests, independent sandbox calibration, and handling sparse advantages and stale batches. A reward formula is not execution or acceleration evidence.
Illustrative example: Synthetic speed scores test update plumbing without timing code.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
How does runtime reward differ from a math verifier?
Reinforcement Learning for Code Optimization
Show answer and example
Answer: Correctness can be relatively deterministic; time depends on inputs, sandbox and load. Use gates and reference percentiles, not one wall-clock observation.
Illustrative example: A busy shared worker can slow unchanged code without a regression.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
Why isn't correctness alone enough for code optimization RL?
Reinforcement Learning for Code Optimization
Show answer and example
Answer: The task also targets speed, but input size and load affect timing. Use dedicated tests and calibration to separate actual improvements from noise.
Illustrative example: Two correct algorithms may show complexity differences only at larger inputs.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Core mechanism
Why not rerun the full decoder for Penelope latent steps?
Penelope: Localized Latent Recurrence for Efficient Structured Reasoning
Show answer and example
Answer: A shared recurrent transform and gated state refine one decoder boundary repeatedly. Only that local computation increases; task gains require measurement and loop count is not reasoning ability.
Illustrative example: Refine a structured problem three times at the boundary without rerunning completed lower layers.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Implementation details
Can K independent layers replace K Penelope updates?
Penelope: Localized Latent Recurrence for Efficient Structured Reasoning
Show answer and example
Answer: No. Recurrence reuses Fθ and GRU/time gates; independent layers add parameters and alter the constraint. The local budget omits the paper's CoT-to-latent curriculum.
Illustrative example: Reusing one block twice differs from running two separately parameterized blocks.
Sections in the Chinese source: 核心公式, 复现边界. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Method comparison
How does local recurrence differ from repeating the whole decoder?
Penelope: Localized Latent Recurrence for Efficient Structured Reasoning
Show answer and example
Answer: It only repeats the designated latent block. Full reruns repeat lower and upper layers as well. Equal step counts do not match compute or information paths.
Illustrative example: Two small block calls cost less than two complete passes.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Basics
What does Penelope repeatedly refine?
Penelope: Localized Latent Recurrence for Efficient Structured Reasoning
Show answer and example
Answer: A continuous hidden state at a local decoder boundary, with shared latent blocks and a gate before higher layers. It does not emit visible text on every loop.
Illustrative example: Refine one hidden position twice instead of writing the answer twice.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why first keep new tasks in an episodic buffer and consolidate only repeated reliable patterns?
Show answer and example
Answer: An isolated event may be noise or exceptional. Episodic storage preserves context; repeated reliable patterns are better candidates for stable parametric knowledge. Self-routing chooses how to use both. Repetition needs reliability checks so recurring mistakes are not consolidated as rules.
Illustrative example: Example: Record the first temporary outage as an event. Only repeatedly confirmed configuration constraints should become a general pattern.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
Does the local parametric route actually extend LLM weight blocks?
Show answer and example
Answer: No. Local parametric memory is a structured pattern table that tests thresholds, episodic-to-consolidated routing, and counts. It neither trains nor expands model weights. Consolidation counts and deterministic success do not establish LLM internalization or streaming capability.
Illustrative example: Example: Adding a rule after three occurrences is a state transition. If model weights did not change, it is not parametric learning.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
What is the trade-off between episodic retrieval and parametric memory for updates and use?
Show answer and example
Answer: Episodic entries are easy to append, audit, and delete, but reads consume retrieval and context. Parametric memory internalizes recurring patterns for direct use, but individual facts are harder to undo. The approaches are complementary; consolidation is not merely copying tables and can accumulate errors.
Illustrative example: Example: Keep a temporary meeting date as traceable episodic memory. A stable tool-use pattern may be worth consolidating and changes less often.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Why can't UniMem handle streaming tasks with fixed weights alone or by accumulating raw episodes indefinitely?
Show answer and example
Answer: Fixed weights cannot promptly absorb new situations. Raw episodes create growing retrieval and context costs. UniMem combines recent examples with stable recurring knowledge to adapt and reuse experience. Timeliness and stability matter, and frequent patterns are not necessarily correct.
Illustrative example: Example: Record a new service's first interface change as a specific event. Only after a rule continues to hold is it a candidate for a stable habit.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why distinguish decomposition, temporal, and recovery edges in a skill graph?
Show answer and example
Answer: Decomposition maps a high-level goal to AtomicOps; temporal edges constrain ordering; recovery edges provide failure paths. A generic similarity edge would lose execution semantics. After retrieving a subgraph, ground it in current state. Finding a skill name is not action completion.
Illustrative example: Example: Upload decomposes into selecting and submitting a file. A failure can follow a recovery edge to refresh login; doing that as a universal prerequisite is unnecessary.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
Does constructing a graph from controlled trajectories show an online LLM learned to extract skill relations?
Show answer and example
Answer: No. Local nodes and relations are supplied by controlled traces, testing hierarchy, subgraph retrieval, and reuse without online LLM extraction. AtomicOp reuse counts show real reuse within the fixture, not semantic discovery in open environments or extrapolated token savings.
Illustrative example: Example: Following a supplied recovery edge tests the graph executor. It does not show a model can infer that edge from an unseen failure log.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
What is the trade-off between retrieving a relevant skill subgraph and placing the whole graph in context?
Show answer and example
Answer: A subgraph limits interpretation and grounding to useful decomposition, order, and recovery relations. The full graph keeps more paths but costs more context and choices. A local subgraph can miss remote prerequisites, so check execution conditions; shorter input is not necessarily better task performance.
Illustrative example: Example: Retrieve the failed upload and nearby recovery nodes. If authentication is absent, the compact subgraph still cannot execute legally.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Why are high-level skills and AtomicOps different node types?
Show answer and example
Answer: High-level skills describe goals and reusable procedures. AtomicOps are executable actions linked through decomposition. Treating both as text snippets blurs abstract plans and immediately executable actions. Ground operations using current state and tool contracts rather than executing a high-level name.
Illustrative example: Example: Organize orders is high level; query, verify status, and write output are atomic operations that can become legal calls after decomposition.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
Why does a solver's value difference between adjacent states complement rather than replace terminal outcomes?
Show answer and example
Answer: Value change provides local progress credit toward solvability and can identify important steps. Solver estimates can be imperfect and local improvement can end in failure, so retain outcome advantage as a global constraint. Any hidden answers available to the training-side solver must be isolated from evaluation actors.
Illustrative example: Simplifying constraints can earn positive local credit, but a later invalid result still makes the trajectory unsuccessful.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
How does CAST avoid reversing states in V(s_{t+1})−V(s_t)?
Show answer and example
Answer: Bind before and after states to the same action and use a consistent value direction. Reversing them records progress as negative credit. Count actual solver calls and keep training-side values from becoming a hidden-answer interface for evaluated policies.
Illustrative example: A value change from 0.2 to 0.5 should give +0.3, not −0.3.
Sections in the Chinese source: 核心公式, 本地复现. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
Do solver state values and a learned policy value critic have the same source of supervision?
Show answer and example
Answer: A solver computes values externally in a solvable environment, while a critic usually learns estimates from experience. Solvability and query costs constrain solvers; accuracy of a tiny shortest-path oracle does not generalize to arbitrary agent tasks.
Illustrative example: Exact solutions on small deterministic graphs do not imply a similarly cheap perfect solver exists for web tasks.
Sections in the Chinese source: 背景与主要改动, 本地复现. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why does CAST query a solver for values of adjacent states instead of relying only on final wins or losses?
Show answer and example
Answer: Terminal signals cannot localize contributions in long games. Solver value before and after an action yields turn-level credit alongside outcome to indicate how each step changed the situation.
Illustrative example: A step can materially improve reachability before completion; its value difference supplies earlier feedback.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why let ranking read SID traces but detach shared outputs and train LoRA?
Unifying Generative Recall and Multi-Objective Ranking in a Single Decoder-Only Sequence
Show answer and example
Answer: Paths supply retrieval evidence. Role-specific masks share forward information, while detached shared outputs and low-rank updates reduce backward interference.
Illustrative example: Rank using outdoor→camping→tent while adjusting only ranking LoRA for new scoring preferences.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
How does ranking LoRA protect the shared generation trunk?
Unifying Generative Recall and Multi-Objective Ranking in a Single Decoder-Only Sequence
Show answer and example
Answer: Stop gradients through shared Q/K/V outputs in the ranking branch and train ranking-specific LoRA. This is not freezing everything or fully sharing gradients. Local scope excludes production beams and parallelism.
Illustrative example: Ranking loss changes LoRA without updating shared generation projections through that route.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
Why do generation and ranking queries need different masks?
Unifying Generative Recall and Multi-Objective Ranking in a Single Decoder-Only Sequence
Show answer and example
Answer: Generation reads only earlier SIDs; ranking can read the full path and synchronized item tokens. Future code access would leak generation targets, while restricting ranking can hide useful evidence.
Illustrative example: Level-two generation cannot read level three, but ranking can.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Why use generated SID traces as a ranking bridge?
Unifying Generative Recall and Multi-Objective Ranking in a Single Decoder-Only Sequence
Show answer and example
Answer: The generated path contains intermediate retrieval choices, beyond a shared encoder or final ID. A heterogeneous decoder reads the trace with item features.
Illustrative example: Rank with the semantic prefixes that led to the item.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why can spectral softening counter collapse?
Show answer and example
Answer: Dominant singular modes concentrate tokens and limit deep interaction. Softening reduces disparity and spectral residuals give attention structural cues. This is not lossless equalization; weak modes can be noise.
Illustrative example: Reduce popularity dominance so genre directions can matter without assuming every weak direction helps.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
What should local SpecFormer SVD tests verify?
Show answer and example
Answer: Validate shapes, nonnegative singular values, reconstruction, positional residual alignment, and stability near zero rank. Public proxies do not reproduce production uplift.
Illustrative example: Correct matrix reconstruction does not show genuine interest recovery.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does spectral softening differ from L2-normalizing embeddings?
Show answer and example
Answer: Unit norms control individual lengths, not collective directional spectrum. Singular-value transforms redistribute subspace energy.
Illustrative example: Unit-length vectors can still all point the same way.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why inspect singular spectra for representation collapse?
Show answer and example
Answer: Concentration in a few directions reduces diversity. Spectral softening changes relative singular values; spectral positional residuals affect attention. A flatter spectrum is not task evidence.
Illustrative example: Nearly parallel user vectors struggle to distinguish interests.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why condition OxygenREC-v2 from the first SID and select post-training constraints by uncertainty?
OxygenREC-v2: Internalizing Discrimination into Generative Recommendation
Show answer and example
Answer: Late reranking cannot recover omitted items. A click/cart/order instruction shapes the whole decoding path. Training-only future behavior gives low-entropy positions confident teacher targets and high-entropy positions forward KL to preserve alternatives. Future events must not enter serving inputs.
Illustrative example: Illustration: An order instruction favors buyable branches from the start; uncertain branches retain several plausible products instead of one hard answer.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Why marginalize the three proxy behaviors in local OxygenREC-v2 instead of selecting the held-out target's class?
OxygenREC-v2: Internalizing Discrimination into Generative Recommendation
Show answer and example
Answer: MovieLens has no click/cart/order. Choosing a condition by the target's popularity leaks identity clues. Marginalizing avoids that hint but changes the original task, so conclusions remain proxy-only. Verify privileged views, entropy routing, and losses separately.
Illustrative example: Illustration: Telling the model in advance the target is low-popularity narrows search; combine the unknown conditions without using the target.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
Why is training with privileged future supervision different from serving with future features?
OxygenREC-v2: Internalizing Discrimination into Generative Recommendation
Show answer and example
Answer: The training teacher can use later behavior to construct richer targets for a student that reads current inputs only. Future events are not yet available online. High-entropy forward KL constrains the distribution; it does not make future inputs legal.
Illustrative example: Illustration: A later lantern purchase can supervise a training example, but feeding it into the test prefix bypasses prediction.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Why put click, cart, and order into OxygenREC-v2 decoder instructions?
OxygenREC-v2: Internalizing Discrimination into Generative Recommendation
Show answer and example
Answer: Goals change which candidates should be generated. Conditioning from the first SID can improve coverage before ranking, while weighted pretraining and post-training connect feedback. Use a legitimate task input, not one inferred from the test target. Local popularity is not real multibehavior supervision.
Illustrative example: Illustration: Cart and casual click can need different first branches; reranking click-heavy candidates may miss order items.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why penalize redundancy in a new Mosaic specialist when several user embeddings already exist?
Mosaic: A Fleet of User Embedding Specialists for Recommendation at Meta
Show answer and example
Answer: New networks can repeat old knowledge while adding serving cost. Mosaic forms a representation fleet, mines task relations to learn joint behavior, and penalizes duplication against existing vectors. The goal is complementary information for downstream readers; geometric orthogonality does not imply business independence.
Illustrative example: Illustration: If old vectors already remember brands, a new specialist could learn when users explore instead of remembering the same brands again.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
Why warm up Mosaic's cosine redundancy loss and evaluate complementarity separately?
Mosaic: A Fleet of User Embedding Specialists for Recommendation at Meta
Show answer and example
Answer: Strong early separation can make untrained features diverge randomly. Warm-up lets task information form before reducing overlap. Lower or negative cosine shows geometry, not useful new evidence. Preserve the local Hit decrease and head-share increase.
Illustrative example: Illustration: Random vectors can be uncorrelated and useless for ranking. First learn meaningful interests, then verify that another specialist adds value.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
What supervision does Mosaic's MRM joint label add beyond independent binary tasks?
Mosaic: A Fleet of User Embedding Specialists for Recommendation at Meta
Show answer and example
Answer: Marginal labels constrain individual frequencies, whereas Cartesian-product labels represent which events co-occur. Related-task mining selects useful combinations, and label space can grow. Joint association does not establish causality; exposure can influence both.
Illustrative example: Illustration: Clicks and long views can each happen half the time together or disjointly. Marginals alone cannot distinguish them.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why isn't Mosaic's specialist fleet equivalent to widening one general user embedding?
Mosaic: A Fleet of User Embedding Specialists for Recommendation at Meta
Show answer and example
Answer: Memory, dense interaction, sequence, and downstream co-training specialists optimize different knowledge and can evolve independently. Downstream consumers select interfaces. One wider model can still be dominated by one objective. A fleet adds storage and maintenance and does not guarantee independence.
Illustrative example: Illustration: A memory specialist recalls brands, a sequence specialist detects this week's trip, and a jointly trained specialist adapts to purchases; width alone need not separate these roles.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Core mechanism
Why embed the item cache as a sparse model parameter table in Memory Layer instead of publishing it externally?
Memory Layer: Train the In-Model Cache for Recommendation Models
Show answer and example
Answer: Fresh tower outputs in training and old cache values at serving differ, and a miss bypass creates a third distribution. Model writeback and reads let training adapt to cache semantics; always-on features score misses. This changes dependencies and train–serve consistency, not guaranteed freshness or zero read cost.
Illustrative example: Illustration: Training on today's item vector but serving yesterday's can drift. An internal cache aligns the read path, while creator or topic features still score misses.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Implementation details
Why is Memory Layer's η=1 writeback not ordinary small-step SGD on M[i]?
Memory Layer: Train the In-Model Cache for Recommendation Models
Show answer and example
Answer: It overwrites the entry with the tower output rather than moving an old value gradually along a gradient. Read from the table, zero the missing component, and retain general attributes. Check write error, coverage, and staleness; calling the table trainable does not override assignment semantics.
Illustrative example: Illustration: If the new output is [1, 2, 3], the entry should equal it after writeback, not move one tenth of the way from the old value.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Method comparison
How does internal writeback change consistency and maintenance versus periodic full external cache publication?
Memory Layer: Train the In-Model Cache for Recommendation Models
Show answer and example
Answer: Coupling writes to training events avoids mismatch from a separate publisher. External rollout gives central control but can lag and miss items. Internal tables still have capacity, concurrency, and versioning concerns; always-on is a miss fallback. The local structure check does not establish Meta throughput.
Illustrative example: Illustration: Training updates item A immediately while unseen items stay old; periodic publishing changes the whole table at once. Hit rate alone cannot rank these schedules.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Basics
Why read the same representation in training, evaluation, and serving for Memory Layer?
Memory Layer: Train the In-Model Cache for Recommendation Models
Show answer and example
Answer: Using fresh tower outputs only in training and old external cache online creates mismatch, with misses another distribution. Training writes the internal table; scoring reads it consistently and retains always-on features on misses. Record missingness and age because consistency does not ensure freshness.
Illustrative example: Illustration: Reading one shared table avoids training on the latest movie vector while serving a different version, but entries can still be stale.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Core mechanism
Why does CORE classify High versus Non-High first, then distinguish Mid and Low only for Non-High?
CORE: A Unified Cascaded Ordinal Relevance Estimation Framework for E-commerce Search
Show answer and example
Answer: Ordered labels have different boundaries; a flat three-way classifier conflates exact matching with looser substitution. Conditional heads specialize, and step rewards apply only to executed steps. High examples must not produce a Mid/Low step gradient. Aggregate teacher probabilities correctly in distillation.
Illustrative example: First check exact model match; only a non-match requires deciding whether it is substitutable or unrelated.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Implementation details
Why does CORE distillation use logsumexp for the Non-High teacher logit?
CORE: A Unified Cascaded Ordinal Relevance Estimation Framework for E-commerce Search
Show answer and example
Answer: Non-High combines Mid and Low probabilities, so use `log(exp z_M + exp z_L)`, not their logit mean. The second head compares only Mid and Low with aligned masks and conditional labels. Copying three logits into both heads changes their meaning.
Illustrative example: Two substantial Mid and Low probabilities can together exceed High; averaging logits misses their combined mass.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Method comparison
How does CORE step-GRPO allocate credit differently from one reward for the final class?
CORE: A Unified Cascaded Ordinal Relevance Estimation Framework for E-commerce Search
Show answer and example
Answer: Final reward cannot identify the first judgment, the conditional second step, or format as the source of error. Step rewards normalize within groups for valid steps, masking Mid/Low after High. Credit depends on verifiable steps, not on longer reasoning text.
Illustrative example: A correct High decision should not force a Mid/Low response or ambiguous updates to a step that never ran.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Basics
Why are CORE's High, Mid, and Low not simply unordered categories?
CORE: A Unified Cascaded Ordinal Relevance Estimation Framework for E-commerce Search
Show answer and example
Answer: High emphasizes strict entity and intent match; Mid versus Low distinguishes acceptable substitutes from unrelated results. Different criteria and error costs motivate conditional heads, but ordering does not imply equally spaced numerical targets.
Illustrative example: The requested phone model is High, a suitable alternative may be Mid, and an unrelated accessory may be Low.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Core mechanism
Why sigmoid each reward margin before averaging by inclusion?
What do Reward Models Memorize?
Show answer and example
Answer: Sigmoid is nonlinear. Convert each actual run's margin into a probability, then separately average seen and held-out runs for memorization and generalization. Their difference can be negative. Group by actual manifests and keep users isolated.
Illustrative example: Large margin differences can be hidden by averaging before probability saturation; retain each run's membership.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Implementation details
Can inclusion in reward-model training be inferred from a high score?
What do Reward Models Memorize?
Show answer and example
Answer: No. Use actual run manifests to assign inclusion and held-out status; scores are outcomes. Inferring inclusion from outcomes is circular. Check user-level isolation. The local aggregator does not create the paper's training runs.
Illustrative example: A high-scoring held-out pair stays held out; do not move it to support a memorization story.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Method comparison
How does CM differ from training accuracy minus test accuracy?
What do Reward Models Memorize?
Show answer and example
Answer: CM compares the same pair across runs and splits by user to avoid leakage. Aggregate split means also mix difficulty differences. CM depends on split and training quality; it is not proof of verbatim memorization per item.
Illustrative example: A train–test gap between different user groups may reflect their difficulty rather than memory of any one preference.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Basics
Why separate in-training and held-out performance in reward-model memorization?
What do Reward Models Memorize?
Show answer and example
Answer: A high score on a seen pair could reflect broad learning or memorization. Compare the same pair across actual runs including and excluding it; the difference measures conditional memorization.
Illustrative example: Compare models trained with this pair to models that did not see it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Core mechanism
Why allow Drop and Untouch instead of applying all three stages to every chunk?
DataOrchestra: Learning to Orchestrate Per-Example Curation of Pretraining Data
Show answer and example
Answer: Clean text can be damaged by costly rewrites. Choose Drop, Untouch, or Clean; only Clean plans noise pruning, surface correction, pedagogical augmentation, and per-chunk instructions. Plans are executed and verified. Optimize quality versus compute, not cleanup count.
Illustrative example: Leave a clean textbook alone, prune navigation on a web page, and drop a fragment missing content.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Implementation details
Can raw teacher plans be used as final SFT targets without verification?
DataOrchestra: Learning to Orchestrate Per-Example Curation of Pretraining Data
Show answer and example
Answer: No. Execute tools and verify results; correct failed rewrites up to a limit and remove stages with lost content or insufficient change. Only verified evolved plans train the orchestrator.
Illustrative example: A format repair that deletes content must not remain a success example.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Method comparison
Why preserve Untouch in sample-level orchestration?
DataOrchestra: Learning to Orchestrate Per-Example Curation of Pretraining Data
Show answer and example
Answer: It explicitly allows no processing, avoiding cleaning rate as a false quality goal. Per-sample plans select stages; one corpus pipeline can overrewrite and waste tools.
Illustrative example: Rewriting correct code may consume compute and damage content.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Basics
Why make chunk-level data decisions?
DataOrchestra: Learning to Orchestrate Per-Example Curation of Pretraining Data
Show answer and example
Answer: Chunks differ in noise and quality. The orchestrator chooses Drop, Untouch, or Clean and organizes operations or instructions only when needed.
Illustrative example: Preserve a clean explanation while removing or repairing corrupted text.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Core mechanism
Why separate debiasing and exploration instead of relying on freshness features?
Breaking the Loop: An Empirical Comparison of Strategies for Novelty and Freshness in YouTube Music
Show answer and example
Answer: Content age does not fix logged exposure selection. IPS and removable bias towers adjust training; SNGP uncertainty gives unknown items exposure online. Both depend on assumptions, and less head concentration does not prove satisfaction.
Illustrative example: A new video may lack clicks because it has never been shown; explore and collect feedback rather than boosting all new videos blindly.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Do local SNGP proxies or negative results explain YouTube's live study?
Breaking the Loop: An Empirical Comparison of Strategies for Novelty and Freshness in YouTube Music
Show answer and example
Answer: No. There is no full SNGP or continual training. Report interventions, logging propensities, and budgets separately. Popularity is not a measured IPS propensity.
Illustrative example: Low popularity does not establish a low historical impression probability.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
How does a removable bias tower differ from retaining recency in serving?
Breaking the Loop: An Empirical Comparison of Strategies for Novelty and Freshness in YouTube Music
Show answer and example
Answer: A tower can absorb training exposure bias then be removed. Recency remains a serving signal. Do not keep adding learned exposure bias as quality.
Illustrative example: Learning old items' logging advantage need not reward them again in deployment.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why separate training debiasing from serving exploration for freshness?
Breaking the Loop: An Empirical Comparison of Strategies for Novelty and Freshness in YouTube Music
Show answer and example
Answer: Historical exposure favors old items. IPS/bias towers change learning signals, while uncertainty exploration changes actual exposures. Age features alone do not fix feedback loops.
Illustrative example: Old songs have more data because they were shown; new songs need actual opportunities.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Core mechanism
Why does Melo ground entities after recommendation and reflect before retrying failures?
Melo: A Production LLM-Powered Music Recommendation Agent
Show answer and example
Answer: Fluent music descriptions can name nonexistent or mismatched entities. Grounding validates outputs against the real catalog before returning them. Reflection uses the failure to revise intent or retrieval rather than arbitrarily regenerating. Catalog validity and preference quality need separate measures.
Illustrative example: Illustration: If a plausible song is absent from the catalog, revisit intent and retrieval instead of forcing a near match or treating any valid song as relevant.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Implementation details
Why does Melo need a threshold and failure state even if post-grounding argmax always yields a valid entity?
Melo: A Production LLM-Powered Music Recommendation Agent
Show answer and example
Answer: Argmax returns some catalog ID even if every score is low. Thresholds detect insufficient evidence, and original entity validity must be logged before mapping. Checking only whether the mapped result exists hides hallucinations. The local path permits one defined retry.
Illustrative example: Illustration: An invented title can map to a real but unrelated movie. Catalog membership after mapping alone misses the error.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Method comparison
How does one reflective retry in Melo trade reliability against unbounded generation?
Melo: A Production LLM-Powered Music Recommendation Agent
Show answer and example
Answer: A bounded retry can repair grounding while controlling cost. Repeated generation can hallucinate again or eventually guess by luck. Trigger on catalog validity and confidence, then assess repaired quality independently. The local freshness increase came with overall regression, so repairs alone do not prove improvement.
Illustrative example: Illustration: Check the catalog and retry a bad film name once. Repeating until any answer matches inflates success and latency.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Basics
Why split Melo into intent, retrieval, recommendation, and explanation nodes?
Melo: A Production LLM-Powered Music Recommendation Agent
Show answer and example
Answer: Each has distinct inputs and failure conditions. Grounding maps generated entities to available items, and feedback decides whether reflection is needed. Staging makes repair traceable. A legitimate entity need not be relevant, and a fluent explanation cannot fix a wrong candidate.
Illustrative example: Illustration: Repair mapping for an invented film title before explaining it; if a real film is irrelevant, inspect intent or retrieval.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why does Dual-purpose SID use SiDec to reconstruct content embeddings?
Show answer and example
Answer: An identity code alone may not supply the high-dimensional content needed downstream. SiDec reconstructs content from short codes for lower I/O while preserving discrete recommendation identity. Compression is lossy, and local small-codebook quality loss must be reported separately from bandwidth gains.
Illustrative example: Three short codes can replace a large vector read, but a decoder cannot recover differences between products that collide in their codes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
Why concatenate separate one-hot columns per code level in SiDec?
Show answer and example
Answer: Identical integers denote different residual-layer codewords. Layer offsets preserve identity before decoding to content width. Keep item and version aligned with reconstruction targets; the local ridge decoder is a scaled-down path, not full reproduction.
Illustrative example: Code 3 at level one and code 3 at level two require different input columns.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
What is the resource trade-off between short SID plus SiDec and a cached dense embedding?
Show answer and example
Answer: Dense caches preserve vectors at storage and I/O cost. Short codes shift per-item storage into indices and shared decoding, adding compute and error. Lower I/O is not necessarily lower end-to-end latency, and a small-codebook retrieval regression remains meaningful.
Illustrative example: A million vectors can become codes, but a decoding matrix on each request may be costly and a small codebook can hurt relevance.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why should a Dual-purpose SID code also reconstruct a content vector?
Show answer and example
Answer: It supports both generative nomination and ranking features. SiDec converts small indices into approximate content, reducing per-item storage and transmission while training the codes not to retain only collaborative identity. Measure quality loss.
Illustrative example: Small codes save bandwidth but cannot reconstruct a waterproofing property they never encoded.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why does EGR jointly train two tasks despite sharing the backbone for item indexing and user queries?
EGR: Embedding-Native Generative Retrieval with a Shared LLM
Show answer and example
Answer: Shared parameters alone do not align retrieval geometry. Item representation learning maintains indexable content; next-item prediction directs queries toward consumption relations. Training one side against a changing other can stale the index, and serving and evaluation must use matching versions.
Illustrative example: If new item vectors rotate while queries stay in old coordinates, nearest neighbors can break despite matching dimensions.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Why verify item-index and query versions after EGR updates its shared backbone?
EGR: Embedding-Native Generative Retrieval with a Shared LLM
Show answer and example
Answer: Updates can change dot-product meaning, so old index vectors may not match new queries. Record rebuild, training, and cutover boundaries and confirm both IRL and NIP run. A shared class name with one loss disabled is not joint training.
Illustrative example: An old item coordinate system and a new user coordinate system can have equal shapes but incompatible scores.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
What deployment trade-off does EGR joint updating make versus freezing the item encoder?
EGR: Embedding-Native Generative Retrieval with a Shared LLM
Show answer and example
Answer: Freezing stabilizes the index but blocks user supervision from refining item geometry. Joint learning can adapt both sides, requiring item refresh and versioned serving. The local negative result should not be replaced by the design promise.
Illustrative example: A new query direction for science fiction may not match an old index; freezing avoids drift but keeps old limitations.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Why jointly optimize item representation learning and next-item prediction in EGR?
EGR: Embedding-Native Generative Retrieval with a Shared LLM
Show answer and example
Answer: Pure content optimization can omit recommendation distinctions. Next-item supervision retains consumption relationships while the item objective anchors semantics. Shared updates reduce separation but can conflict and do not guarantee alignment.
Illustrative example: Similar-genre movies can appeal to different audiences; joint learning can preserve that difference beyond text similarity.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Core mechanism
Why check fresh-content survival at every stage in PinEqualizer instead of boosting it only in fine ranking?
PinEqualizer: Full Funnel Content Exploration and Debiasing System at Pinterest
Show answer and example
Answer: Items filtered in corpus or retrieval never reach the ranker. Track stage inputs and survival to locate bottlenecks. Content features, interaction dropout, and calibration reduce dependence on old exposure; separate exploration and uncertainty rewards collect feedback. Relevance still constrains exploration.
Illustrative example: Illustration: A new recipe filtered before retrieval cannot benefit from a ranking boost. Preserve entry and explore promising items selectively.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Implementation details
Why preserve validation's UCB α=0 rather than tune α for a better fresh metric?
PinEqualizer: Full Funnel Content Exploration and Debiasing System at Pinterest
Show answer and example
Answer: Validation rejected extra exploration under the current objective. Do not reselect on test. The debiased model itself can raise fresh exposure without UCB. Report freshness denominators and overall regression instead of changing objectives afterward.
Illustrative example: Illustration: If the model already increases fresh impressions severalfold and UCB hurts relevance, do not force α above zero to claim bandit use.
Sections in the Chinese source: 本地复现. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Method comparison
Why does PinEqualizer UCB consider uncertainty and relevance instead of merely lowering popular scores?
PinEqualizer: Full Funnel Content Exploration and Debiasing System at Pinterest
Show answer and example
Answer: Popularity penalties do not identify worthwhile unknown items. UCB promotes low-exposure or uncertain items, with search relevance limiting unrelated exploration. Locally fresh exposure rose but overall NDCG fell, so report both.
Illustrative example: Illustration: Explore a relevant new recipe, not unrelated news just because it is obscure.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Basics
Why treat PinEqualizer exploration as both a supply and a model problem?
PinEqualizer: Full Funnel Content Exploration and Debiasing System at Pinterest
Show answer and example
Answer: New items can disappear in corpus, retrieval, or ranking. Content reduces dependence on old interactions, an exploration corpus allows entry, and uncertainty allocates feedback. Stage survival locates bottlenecks; a high fresh quota does not guarantee relevance.
Illustrative example: Illustration: A favorable ranker cannot recommend an item that retrieval never includes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why does BARGE both restore item boundaries and rerank accumulated semantic paths?
Bridging the Structural Gap: Adapting Autoregressive Generation for Recommendation
Show answer and example
Answer: Ordinary sequence attention can treat the multiple SID codes of one item as separate events; ICA aggregates within-item context and gates it back into each code. HPR evaluates the accumulated generated path to limit propagation of early beam mistakes, while complementary quantization gives alternate decomposition views. These address boundaries, search errors, and diversity respectively.
Illustrative example: Province, city, and street are one address, not three purchases. If the coarse province is wrong, refining the street cannot fix it, so check the path at every level.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Why does BARGE's HPR compare the accumulated path vector instead of only the latest code?
Bridging the Structural Gap: Adapting Autoregressive Generation for Recommendation
Show answer and example
Answer: The cumulative sum includes every choice from coarse to fine and reveals whether the full semantic path fits the user. During inference combine generation log-probability with reranking; during training use contrastive comparisons over cumulative paths. DPD separately constrains projection orthogonality; it is not the same loss or proof of full independence.
Illustrative example: An entry-level code makes sense on its own, but the path has already entered home goods while the user wants a camera. Only the accumulated path exposes the mismatch.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
How do BARGE's two complementary decoding paths differ from widening a single beam?
Bridging the Structural Gap: Adapting Autoregressive Generation for Recommendation
Show answer and example
Answer: A wider beam searches more paths within one encoding structure. Two orthogonal subspaces construct different representations and quantizations, so one early bias need not dominate both. HPR then scores cumulative paths. Extra paths add training and merge costs, and output count alone does not prove complementarity or independence.
Illustrative example: A wider beam may still explore a mistaken coarse code, while a second subspace can recover a missed book if it preserves different semantic cues.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Why do multiple SID tokens for one BARGE item require explicit item context?
Bridging the Structural Gap: Adapting Autoregressive Generation for Recommendation
Show answer and example
Answer: They are decompositions of the same item, but ordinary sequence attention may treat them as independent events. ICA aggregates within-item context and gates it back into each token, joining whole-item meaning to local detail. This addresses representation structure, not the separate task of learning preferences across items.
Illustrative example: A book's topic, subcategory, and collision code describe one book, not three successive reads.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why can shrinking only Windowed-MTP's draft context save reads without changing the target distribution?
Windowed-MTP: Removing the Full-Context Draft-KV Tax at Million-Token Context
Show answer and example
Answer: Draft reads sinks and recent tokens; the full target still sees all history and corrects proposals. The window affects draft cost and acceptance, not the exact target definition. Cutting target context changes the guarantee; local latency does not transfer to production.
Illustrative example: Target can reject a wrong draft caused by missing old evidence, provided its own history stays complete.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Should overlaps between sinks and recent tokens duplicate KV?
Windowed-MTP: Removing the Full-Context Draft-KV Tax at Million-Token Context
Show answer and example
Answer: No. Use the union without duplicates. Only draft indices change; target retains the full cache.
Illustrative example: A position in both sets appears once in draft attention.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Method comparison
Why doesn't a large reduction in draft keys yield the same TPOT reduction?
Windowed-MTP: Removing the Full-Context Draft-KV Tax at Million-Token Context
Show answer and example
Answer: Draft forwards and target checks still cost time, and time per output depends on accepted length AL. Lower acceptance can offset savings.
Illustrative example: Cheaper verification that accepts fewer tokens may yield little throughput gain.
Sections in the Chinese source: 核心公式, 本地复现. The example illustrates the mechanism.
Basics
Why window only the draft?
Windowed-MTP: Removing the Full-Context Draft-KV Tax at Million-Token Context
Show answer and example
Answer: Draft KV reads can bottleneck long contexts. Retain sinks and recent draft history while the full target verifies, changing proposal cost and acceptance.
Illustrative example: Draft might read recent 64 and a few sinks while target reads all 16K.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Core mechanism
Why separate retention and writing rather than use complementary gates in Naju?
Show answer and example
Answer: A complementary write would shrink whenever retention is strong. Independent gates on native discrete dynamics can preserve old state and write new evidence together. Token-dependent directions, convolution, and feedthrough still matter.
Illustrative example: Keep a name while adding age instead of erasing the name in proportion to the new write.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Implementation details
How should preserve-first initialization set Naju's two gates?
Show answer and example
Answer: forget_bias=5 and write_bias=−2 create strong initial retention and weak writes. Retain token-dependent B/C and direct feedthrough.
Illustrative example: Sigmoid(5)≈0.993 and sigmoid(−2)≈0.119 are not equal gates.
Sections in the Chinese source: 核心公式, 本地复现. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Method comparison
How do Naju's native discrete poles differ from Mamba discretizing continuous dynamics?
Show answer and example
Answer: Naju directly parameterizes discrete retention. Mamba derives a transition from continuous dynamics and input-dependent time steps. Both recur but formulas and gate coupling differ.
Illustrative example: One learns the retained fraction per step directly; the other derives it from a time step and continuous matrix.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Basics
Why independently control old-state retention and new writes?
Show answer and example
Answer: A shared complementary gate creates a tradeoff. Separate retain/write gates in the discrete transition let both adjust independently.
Illustrative example: Reading an important new fact need not erase an old fact.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Core mechanism
How does Möbius position encoding differ from standard RoPE across a training window?
Anti-Periodic Positional Encoding: Möbius Boundary Conditions Make In-Context Retrieval Reliable
Show answer and example
Answer: Antiperiodic frequencies on some heads yield −I at one window and I at two. Other heads retain standard RoPE for language modeling. This changes positional geometry, not accessible history or guaranteed retrieval at all lengths.
Illustrative example: At N positions the selected heads reverse phase; at 2N they realign. Actual retrieval still depends on Q/K and training.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Implementation details
Should Möbius RoPE change N dynamically with inference length?
Anti-Periodic Positional Encoding: Möbius Boundary Conditions Make In-Context Retrieval Reliable
Show answer and example
Answer: No. N is fixed training context; local N=64 replaces one of four heads. Dynamic N changes frequencies for identical positions and breaks the boundary definition. Retain negative local results.
Illustrative example: Length-32 and length-80 inputs both use the same training N.
Sections in the Chinese source: 核心公式, 复现边界. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Method comparison
What explicit frequency boundary does Möbius add?
Anti-Periodic Positional Encoding: Möbius Boundary Conditions Make In-Context Retrieval Reliable
Show answer and example
Answer: Set some two-dimensional frequencies to π(2i+1)/N, producing −I across N and I across 2N. Other heads keep standard RoPE. This is geometry, not measured improvement.
Illustrative example: Reverse phase after one window and recover after two, not arbitrary scaling across every head.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Basics
What retrieval instability is Möbius RoPE intended to address?
Anti-Periodic Positional Encoding: Möbius Boundary Conditions Make In-Context Retrieval Reliable
Show answer and example
Answer: Standard RoPE long-range needle retrieval can vary strongly by training seed. Antiperiodic frequencies on some heads aim to stabilize cross-window phase while leaving others unchanged.
Illustrative example: Identical needles in long text can produce very different retrieval results across seeds.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Core mechanism
Why give hard-to-gzip blocks long-range edges, and are they necessarily important?
Parameter-free Adaptive Sparse Attention via Compression-Based Content Selection
Show answer and example
Answer: Blocks above the sample's average compression-ratio threshold are treated as literal-rich and connected at range while preserving local windows and causality. Compressibility is a training-free proxy, not semantic gold; random noise can also be incompressible and too many literal blocks make edges costly.
Illustrative example: An unusual ID may need cross-section access while a repeated template compresses easily, but incompressible gibberish need not help.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Implementation details
Why intersect literal, local, and self edges with the causal mask?
Parameter-free Adaptive Sparse Attention via Compression-Based Content Selection
Show answer and example
Answer: Their union expresses relevance, not chronology. Intersect it with causality and assign the appropriate masks to the three head types. Masking dense local scores does not accelerate a sparse kernel.
Illustrative example: Selecting two literal blocks cannot let the early one read future tokens in the other.
Sections in the Chinese source: 核心公式, 复现边界. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Method comparison
How does a gzip mask vary compared with a fixed BigBird or Longformer pattern?
Parameter-free Adaptive Sparse Attention via Compression-Based Content Selection
Show answer and example
Answer: Fixed masks use predetermined positions; gzip uses per-example block compressibility for literal edges and retains local windows. It avoids learned mask weights but pays for compression and construction, so edge counts do not establish speed.
Illustrative example: Moving a unique passage can move gzip edges without changing a positional pattern.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Basics
Why compress raw byte blocks first?
Parameter-free Adaptive Sparse Attention via Compression-Based Content Selection
Show answer and example
Answer: Compression ratio is a training-free proxy for unique byte density and guides long-range edges, not a semantic importance label.
Illustrative example: Repeated templates compress differently from unique IDs, but hard-to-compress gibberish can be useless.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Core mechanism
Why give more compute to AdaDSF layers with lower input/output similarity?
Adaptive Depth Sparse Framework: Similarity-Driven Resource Allocation for Pre-Trained LLMs
Show answer and example
Answer: Lower cosine suggests a stronger representation transformation. Dense calibration maps layer differences to a globally corrected budget. Layer routers choose tokens and others bypass residually; allocation and alignment training serve different roles.
Illustrative example: With fixed total budget, run the more transformative layer on more tokens instead of uniformly skipping half in each.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Implementation details
Should tokens outside layer top-k be removed?
Adaptive Depth Sparse Framework: Similarity-Driven Resource Allocation for Pre-Trained LLMs
Show answer and example
Answer: No. Bypass the block with the residual and merge in original order. Correct layer ratios to the global budget. Dropping/reordering positions changes later attention and teacher alignment.
Illustrative example: Token five can skip one layer but stays at position five for the next.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Method comparison
What controls should AdaDSF share with Uniform Mixture-of-Depths?
Adaptive Depth Sparse Framework: Similarity-Driven Resource Allocation for Pre-Trained LLMs
Show answer and example
Answer: Match global budget, teacher initialization, alignment steps, and optimizer. Vary layerwise allocation. The local negative result shows adaptation is not necessarily better.
Illustrative example: Comparing 80% and 50% retention confounds strategy with compute.
Sections in the Chinese source: 背景与主要改动, 本地复现. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Basics
Why not give each AdaDSF layer the same token budget?
Adaptive Depth Sparse Framework: Similarity-Driven Resource Allocation for Pre-Trained LLMs
Show answer and example
Answer: Layers transform inputs by different amounts. Dense-teacher similarities allocate more to stronger transformations; routers and sparse paths then align with the teacher.
Illustrative example: A nearly identity layer may need fewer active tokens than one that reorganizes features.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Core mechanism
Why subtract a moving outcome baseline rather than add both rewards directly?
Show answer and example
Answer: Process ratings often track correctness, so direct sums can count success twice. The moving residual attempts to isolate extra reasoning benefit. It depends on checklist and baseline quality, and local proxies don't establish full reasoning performance.
Illustrative example: If correct answers routinely earn high process scores, only an improvement beyond the usual outcome level gives new evidence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Implementation details
Does local TCR call a model to generate per-example checklists?
Show answer and example
Answer: No. Candidate policies test residual computation, not real checklist generation or free-text training. Update EMA in training order, not from test performance.
Illustrative example: A hand-provided checklist does not validate automatic checklist quality.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Method comparison
How does TCR residual differ from adding process and outcome rewards?
Show answer and example
Answer: Addition accumulates both signals. Subtracting a dynamic outcome baseline reduces duplicate credit. It does not prove exact decorrelation or remove judge bias.
Illustrative example: A higher outcome baseline reduces extra credit for an unchanged process score.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Basics
Why subtract outcome EMA from TCR process scores?
Show answer and example
Answer: Process and outcome can double-reward success. The residual captures improvement not already explained by outcome before combining them. EMA is training state, not item truth.
Illustrative example: A correct answer should not automatically earn the same success signal again through process scores.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Core mechanism
Why balance hard-negative discrimination and anchoring?
Co-Evolving LLM Evaluators and Policies via DynamicRubric
Show answer and example
Answer: Fixed rubrics can stop separating improved replies; dynamic dimensions help. Discrimination alone can learn stylistic shortcuts, so an anchor stabilizes the reference. Coevolution is not objective improvement without independent tests.
Illustrative example: When formatting saturates, new criteria should not simply favor verbosity unchecked.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Implementation details
Why tie DynamicRubric weights to prompt and candidate version?
Co-Evolving LLM Evaluators and Policies via DynamicRubric
Show answer and example
Answer: Both determine scoring and new candidates may imply new criteria. Log rubric and anchor versions; don't compare rewards under undeclared weight changes.
Illustrative example: Score changes across two rounds can reflect changed criteria, not actor gains.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Method comparison
How do discrimination and anchoring pull DynamicRubric in different directions?
Co-Evolving LLM Evaluators and Policies via DynamicRubric
Show answer and example
Answer: Discrimination increases sensitivity to current hard cases; anchors retain prior standards and deter co-adaptation. Either alone can reward irrelevant differences or preserve stale criteria.
Illustrative example: Length is easy to distinguish, but an anchor should constrain it if unrelated to quality.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Basics
Why build DynamicRubric criteria from the current candidate set?
Co-Evolving LLM Evaluators and Policies via DynamicRubric
Show answer and example
Answer: Old rules can saturate after policy gains. New hard negatives need discriminating dimensions and weights, while anchors limit judge drift. Not permission to change rules after outcomes arbitrarily.
Illustrative example: Once every answer meets formatting, look for evidence that still separates quality.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Core mechanism
How can GaugeQuant change coordinates without changing the floating-point function?
GaugeQuant: Online Learning of Quantization-Optimal Bases from LLM Symmetries
Show answer and example
Answer: Valid symmetries apply an orthogonal basis on one side and its inverse on the other. Quantization depends on coordinate distributions, so equivalent bases can yield different outliers/errors. Learn quantization-friendly bases only at valid locations.
Illustrative example: Spread an extreme channel across several axes while inverse transformation preserves the floating-point output.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Implementation details
How distinguish functional equivalence, fake quantization, and actual int4 execution?
GaugeQuant: Online Learning of Quantization-Optimal Bases from LLM Symmetries
Show answer and example
Answer: Check orthogonality and equivalent transforms, then quantization and gradients. STE fake quant can use floating kernels. The local work has no int4 kernel, so no throughput or bandwidth conclusion.
Illustrative example: Sixteen levels do not imply bit packing or int4 instructions.
Sections in the Chinese source: 核心公式, 复现边界. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Method comparison
How does GaugeQuant's optimization timing differ from offline calibration?
GaugeQuant: Online Learning of Quantization-Optimal Bases from LLM Symmetries
Show answer and example
Answer: Train bases and outlier objectives with the LM. Offline methods commonly freeze the model and calibrate from samples later. Bit width alone hides training and data differences.
Illustrative example: Both can be W4A4 while one learns a basis during training and the other calibrates afterward.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Basics
Why can activation outliers hurt typical values during quantization?
GaugeQuant: Online Learning of Quantization-Optimal Bases from LLM Symmetries
Show answer and example
Answer: Fixed bit width must cover the range; large channels enlarge spacing for ordinary values. GaugeQuant learns an orthogonal basis with an outlier objective. Floating-point equivalence need not preserve quantization error.
Illustrative example: A few levels spanning 1, 2, and 1000 struggle to distinguish 1 from 2.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why assign global and query-conditioned value codes under one semantic prefix?
TSGR: Taobao Search Generative Retrieval
Show answer and example
Answer: Semantics groups relevance, while value differs overall and by query. Parallel codes and joint VRM combine generator state with item details without a disconnected pre-ranker. Public rating proxies are not transaction value.
Illustrative example: A middling camera accessory may be especially valuable for macro photography.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
What leakage must local TSGR IDs and VRM avoid?
TSGR: Taobao Search Generative Retrieval
Show answer and example
Answer: Use training-period value estimates and code assignments, align query conditions with item identity, and update VRM jointly under protocol. Never encode test conversion.
Illustrative example: Putting a future order amount into an item code leaks the prediction target.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
Why retain global and query-conditioned value codes?
TSGR: Taobao Search Generative Retrieval
Show answer and example
Answer: Global summarizes broad value; conditional code expresses value for specific demand. Global can favor head items and conditional needs reliable query statistics.
Illustrative example: A bestseller may be worse for a specialist query than a niche product.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Why encode business value in TSGR generation?
TSGR: Taobao Search Generative Retrieval
Show answer and example
Answer: Semantic-only retrieval followed by isolated pre-ranking separates retrieval from value. Global and query-conditioned codes plus joint VRM connect generation states with fine business ranking.
Illustrative example: Among semantically similar items, include transaction fit for the current query.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why predict creative lift over a base ad and penalize pixel use in PinDCO?
PinDCO: Whole-Page Aware Dynamic Creative Optimization at Scale
Show answer and example
Answer: Base scores already capture product–user match; increments isolate predicted image, title, and layout effects. Tall creatives consume more feed space, so a pixel adjustment weighs value against footprint. Exploration gathers new feedback. The local pixel-free proxy is not causal layout evidence.
Illustrative example: Illustration: A taller image can gain clicks but displace content. Prefer it only when added value offsets space cost.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Why preserve PAM clipping and the precise definition of relative aspect ratio r?
PinDCO: Whole-Page Aware Dynamic Creative Optimization at Scale
Show answer and example
Answer: P(r)=clip(1−tanh(k(r−1)),0,1) penalizes tall creatives within a legal range and multiplies base objective O. Inverting the ratio or omitting clipping changes the behavior. Local content norm is a pixel-free proxy, not real layout efficiency.
Illustrative example: Illustration: If r means height over width, using width over height reverses the screen-space penalty.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
How does PinDCO ε-greedy help new creatives compared with giving all traffic to historical winners?
PinDCO: Whole-Page Aware Dynamic Creative Optimization at Scale
Show answer and example
Answer: Greedy exposure starves new creatives of evidence. Limited random exploration provides less-selected feedback for component learning, but costs traffic and does not remove all confounding. Pixel adjustment separately controls space; retain catalog and factual safeguards.
Illustrative example: Illustration: A new title with no clicks may simply lack exposure. Explore modestly without showing clearly misleading copy.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why use component-specific towers for images, titles, and layout in PinDCO?
PinDCO: Whole-Page Aware Dynamic Creative Optimization at Scale
Show answer and example
Answer: Components differ in modality and statistics. Separate encoders retain roles before combining them to predict increments for one base ad. Interaction remains necessary to score combinations. A predicted lift is not randomized causal evidence.
Illustrative example: Illustration: Changing a title and changing image shape have different effects; an attractive image can still mismatch the copy.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
Why sample with hints but optimize and infer without them?
Off-Context GRPO: Learning to Reason on Hard Problems using Privileged Information
Show answer and example
Answer: Hints increase successful trajectories and shift behavior. Correct using unhinted target versus hinted behavior ratio. Dropping denominator or retaining hints at evaluation changes the claimed task.
Illustrative example: A training draft can help collect a solution; the student and test still see only the problem.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
What contexts condition numerator and denominator in Off-Context GRPO?
Off-Context GRPO: Learning to Reason on Hard Problems using Privileged Information
Show answer and example
Answer: Numerator uses target probability on the original prompt; denominator uses actual behavior probability with privileged context. Same reply prefix, different contexts; don't add the hint to both.
Illustrative example: Strip hints only from target scoring, not from the sampled behavior's probability.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
How does Off-Context GRPO differ from SFT on hinted replies?
Off-Context GRPO: Learning to Reason on Hard Problems using Privileged Information
Show answer and example
Answer: It retains verifiable rewards and relative group signals and corrects hinted behavior versus unhinted target. Imitation doesn't automatically correct that shift. Both must isolate evaluation gold.
Illustrative example: A hint makes a path common without implying the unhinted model originally sampled it often.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
Why let only Off-Context GRPO behavior see privileged hints?
Off-Context GRPO: Learning to Reason on Hard Problems using Privileged Information
Show answer and example
Answer: Hints help collect successes while the target sees the original prompt as deployed. Letting target see the solution evidence would change the task.
Illustrative example: A sampler can see a draft; omit it when scoring the target.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
How does fixed-spectrum ISO differ from ordinary low-rank updates?
ISO: An RLVR-Native Optimization Stack
Show answer and example
Answer: It adjusts input/output singular directions while retaining magnitudes, rather than adding arbitrary low-rank changes. This constrains reachable weights and structures expert merges. Small tests don't establish full model optimization.
Illustrative example: Equal singular values with rotated directions can implement different mappings.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
How avoid training Σ in ISO?
ISO: An RLVR-Native Optimization Stack
Show answer and example
Answer: Keep Σ0 fixed and optimize only permitted input/output frames, checking structure. Free matrix updates change spectrum. Local candidate tests don't cover the full LLM stack.
Illustrative example: Unrestricted gradients on a reconstructed weight bypass fixed singular values.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
Are ISO fixed spectrum and GCPO protected subspaces the same?
ISO: An RLVR-Native Optimization Stack
Show answer and example
Answer: No. ISO fixes values and changes frames; GCPO excludes updates into selected principal directions. They fix different quantities and allow different changes.
Illustrative example: Rotating directions at fixed scale isn't holding principal directions fixed and using their complement.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
Why freeze singular values but adjust singular frames in ISO?
ISO: An RLVR-Native Optimization Stack
Show answer and example
Answer: Fixed spectral strength and rotated input/output directions constrain adaptation without freezing the whole layer or keeping function unchanged.
Illustrative example: The same stretches in rotated bases can act on different features.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Core mechanism
Why must repair proposals not automatically replay arbitrary external tools from imported logs?
Show answer and example
Answer: Logs are observations and can be stale, unauthorized, or side-effecting. Separate diagnosis and attribution from authorized execution. Layer hypotheses and suggestions over immutable traces, and create independent rerun evidence after a human or policy gate. Reference-privileged diagnosis is distinct from answer-free online ability.
Illustrative example: A recorded command deleting a temporary folder should not run on import. Suggest a repair, then let authorized execution decide.
Sections in the Chinese source: 背景与主要改动, 核心机制与评测边界. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Implementation details
May AgentDebugX automatically replay tools from logs after proposing repairs?
Show answer and example
Answer: No. Real reruns need a human or policy gate. Imported logs do not authorize arbitrary external commands. Score a real independent post-repair trajectory for loop evidence. Paper results did not evaluate memory or Error Hub gains.
Illustrative example: If a log contains a deletion, the debugger may propose a correction rather than rerunning the command.
Sections in the Chinese source: 背景与主要改动, 核心机制与评测边界. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Method comparison
What comparison boundary does the reference answer in Who&When introduce?
Show answer and example
Answer: The diagnostic receives the task answer but not the responsible agent or step, so it is localization with ground truth. Do not omit that privilege and present it as an answer-free online capability test.
Illustrative example: Knowing the intended result helps find deviations but does not identify who first erred.
Sections in the Chinese source: 背景与主要改动, 核心机制与评测边界. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Basics
Why does AgentDebugX preserve immutable raw traces and layer diagnoses on top?
Show answer and example
Answer: Failure analysis needs the original sequence and evidence; overwriting logs hides behavior. Detect, Attribute, Recover, and Rerun produce separately traceable artifacts. The symptom step is not necessarily the cause.
Illustrative example: A tool error late in a trace may originate from bad arguments supplied earlier.
Sections in the Chinese source: 背景与主要改动, 核心机制与评测边界. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Core mechanism
Why put gates, independent validation, and rollback outside the RecEvolve research agent?
RecEvolve: A Knowledge-Driven Autonomous Agent System for Recommender Systems
Show answer and example
Answer: Agents can propose useful mutations or exploit a metric. External gates reject invalid candidates, then independent validation compares against the champion. Atomically promote only evidence-backed gains; otherwise retain the incumbent and record failure. Keep test out of the search loop.
Illustrative example: Illustration: Reject an illegally sharp low-temperature candidate; keep the champion if another candidate runs but scores worse on validation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Implementation details
Why finish validation artifacts before atomically updating the RecEvolve champion pointer?
RecEvolve: A Knowledge-Driven Autonomous Agent System for Recommender Systems
Show answer and example
Answer: Incomplete training, missing metrics, and interrupted writes must not overwrite a working incumbent. Verify terminal state and J before switching the pointer. No promotion is a valid outcome, and small local effects do not establish online gains.
Illustrative example: Illustration: A half-written checkpoint should leave the champion pointing to the old complete model.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Method comparison
Why can an external critic gate reduce metric gaming compared with agent self-evaluation?
RecEvolve: A Knowledge-Driven Autonomous Agent System for Recommender Systems
Show answer and example
Answer: Self-review can favor proposals or shortcuts. An executable external gate fixes allowable changes and independent validation decides promotion. It governs evidence, not innovation, and test cannot feed routine search.
Illustrative example: Illustration: Reject an extreme temperature by a fixed constraint rather than accept the agent's assurance it did not cheat.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Basics
Why aren't RecEvolve's knowledge base and runnable model artifacts interchangeable?
RecEvolve: A Knowledge-Driven Autonomous Agent System for Recommender Systems
Show answer and example
Answer: The knowledge base stores proposals, protocols, findings, and failures for later choices. Artifacts store parameters and configuration for execution. Promotion evidence must link to the specific runnable version.
Illustrative example: Illustration: 'The content branch won' cannot be rerun without config, and bare weights do not show how they beat the champion.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Core mechanism
Why keep a separate RAMP path for unavailable personalization instead of filling zeros?
Show answer and example
Answer: Zeros conflate unavailable features with true zeros and force one representation across distinct inputs. An availability mask selects personalized or public paths, and training distills rich predictions to the restricted path. Serving remains limited to permitted features and cannot recover hidden personal history.
Illustrative example: Illustration: An unconsented request uses content and context, while authorized training examples can provide soft supervision without exposing restricted fields online.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Implementation details
Why derive RAMP availability masks from actual permissions rather than prediction quality?
Show answer and example
Answer: The mask expresses feature access, not labels or which branch wins afterward. Distill on authorized samples and keep restricted test traffic free of personal features. Distinguish unavailable values from legitimate zeros.
Illustrative example: Illustration: Selecting the private branch only when a purchase occurs leaks outcomes. Use permission and field availability at request time.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Method comparison
What does RAMP distillation add beyond training on missing features alone?
Show answer and example
Answer: Missing-feature training uses hard labels. A rich teacher adds soft relational judgments, but student inputs may not recover all information and bias can transfer. A large relative gain over a tiny local baseline is not definitive industrial robustness.
Illustrative example: Illustration: A teacher can distinguish two ads with similar public features; a student may learn a trend but cannot distinguish every user without public clues.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Basics
Does RAMP's non-personalized branch predict the same thing for every request?
Show answer and example
Answer: No. It can use ad content, public attributes, and request context without restricted personal fields. Constant output would discard allowed evidence. Follow permissions and do not disguise personal IDs as public.
Illustrative example: Illustration: Rain gear on a rainy day and food on a sunny day can score differently without private browsing history.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Core mechanism
Why does PPL-Factory vary the selection band by task and budget instead of always choosing hardest?
PPL-Factory: Task-Aware and Budget-Aware Data Selection from Language Modeling to Reasoning
Show answer and example
Answer: Difficulty and usefulness are not monotone. A frozen model scores under the task protocol, and easy, middle, or random middle sampling trade learning signal and coverage. Reasoning SFT ranks response loss only.
Illustrative example: With dozens of examples, hardest-only can lack useful signal; random midrange may retain coverage, while larger budgets differ.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Implementation details
Why match frozen scorer and starting checkpoint across subsets?
PPL-Factory: Task-Aware and Budget-Aware Data Selection from Language Modeling to Reasoning
Show answer and example
Answer: Changing score model or initialization confounds selection. Match block count and tuning budget; preserve negative transfer of middle selection and do not retune on test.
Illustrative example: Random, Easy, and Middle must start from the same checkpoint to attribute PPL differences.
Sections in the Chinese source: 本地复现. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Method comparison
Can response-only and prompt-plus-response NLL ranks differ?
PPL-Factory: Task-Aware and Budget-Aware Data Selection from Language Modeling to Reasoning
Show answer and example
Answer: Yes. Full sequence includes prompt length and predictability; response-only isolates reasoning/answer tokens. Packed-block LM scoring is another protocol.
Illustrative example: A long easy prompt can hide a difficult answer in aggregate NLL.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Basics
Why tie example selection to task and budget?
PPL-Factory: Task-Aware and Budget-Aware Data Selection from Language Modeling to Reasoning
Show answer and example
Answer: No fixed easiest/hardest policy suits all budgets. Task-specific frozen-model NLL guides easy, middle, or mid-random selection.
Illustrative example: A narrow band at tiny budget may hurt coverage; random sampling within the middle pool offers another trade-off.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Core mechanism
Why add three-tap convolution after Q/K/V despite having attention?
Convolution for Large Language Models
Show answer and example
Answer: Attention selects history by content. Short depthwise residual convolution adds nearby ordered patterns to projected Q/K/V before attention, without replacing global interaction. Left padding preserves causality.
Illustrative example: Local context can combine negation with already seen neighboring words before matching faraway context, but cannot read an unseen word.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Implementation details
How should QKV residual convolution handle padding, activation, and bypass?
Convolution for Large Language Models
Show answer and example
Answer: Use causal left padding, add the convolution to projected Q/K/V, then run attention. Do not insert normalization or activation into this documented bypass.
Illustrative example: Width three at t uses t, t−1, t−2, not symmetric future padding.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Method comparison
How does depthwise QKV convolution differ from replacing attention with convolution?
Convolution for Large Language Models
Show answer and example
Answer: It enriches projections without removing global attention. Depthwise kernels do not fully mix channels, keeping overhead small, and attention still serves long-range access.
Illustrative example: Combine three adjacent positions, then still read a name hundreds of words earlier.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Basics
What information preference does Conv-LLM add to attention?
Convolution for Large Language Models
Show answer and example
Answer: Global matching accesses distant tokens without inherently emphasizing local word combinations. Short convolution first enriches Q/K/V with local order, while attention handles global interaction.
Illustrative example: Aggregate nearby words in “not very good” before reading broader context.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Core mechanism
Why jointly train compression and concatenation reuse in C²KV?
C²KV: Compressed and Composable KV Cache Reuse for Efficient LLM Inference
Show answer and example
Answer: Independently compressed KV can depend on original positions or context. Compression tokens and structured attention train representations that work together across segments. This still has information loss and interface constraints.
Illustrative example: Two compressed documents should jointly supply evidence, not only reconstruct individually.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Implementation details
What interface checks matter between Extractor output and concatenated input?
C²KV: Compressed and Composable KV Cache Reuse for Efficient LLM Inference
Show answer and example
Answer: Verify per-segment Z shape, concatenation boundaries, and structured attention visibility. Local mini-suite omits paper-scale training and kernels; concatenability alone is insufficient.
Illustrative example: Correct lengths with wrong boundaries can cause reads from the wrong segment.
Sections in the Chinese source: 核心公式, 复现边界. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Method comparison
How does joint training differ from compressing each segment and arbitrarily concatenating?
C²KV: Compressed and Composable KV Cache Reuse for Efficient LLM Inference
Show answer and example
Answer: Joint training accounts for position-independent composition. Independent compression can lose cross-segment evidence. Equal compression ratios do not imply equal concatenation behavior.
Illustrative example: Two fragments can answer local questions individually yet fail cross-fragment queries when combined.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Basics
Why make compressed C²KV both short and reusable by concatenation?
C²KV: Compressed and Composable KV Cache Reuse for Efficient LLM Inference
Show answer and example
Answer: Short single-segment cache does not guarantee combined use. Train compression tokens and structured attention for both compression and reuse.
Illustrative example: Reuse several compressed fragments on a new query rather than prefill them again.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why exchange Wukong and HSTU state bidirectionally at every layer rather than only at the end?
WHALE: A Scalable Unified Model for Recommendation with Wukong-HSTU Architecture
Show answer and example
Answer: Field interactions model high-order combinations and sequences capture evolving behavior. Cross-attention lets each guide later processing by the other. This adds dependencies and cost, not reusable user-only caching; local evidence is scaled public evaluation.
Illustrative example: Weekend context can focus sequence attention on camping, then updated interests inform price/category interactions.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
What update order matters for WHALE's two streams?
WHALE: A Scalable Unified Model for Recommendation with Wukong-HSTU Architecture
Show answer and example
Answer: Follow same-layer query/key/value inputs as specified; accidental in-place overwrites change what the other stream reads. Public code lacks industrial features and scaling evidence.
Illustrative example: Updating x before h can differ from both reading old layer states.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does bidirectional cross-attention differ from late fusion?
WHALE: A Scalable Unified Model for Recommendation with Wukong-HSTU Architecture
Show answer and example
Answer: Late-fusion paths stay independent until the end. Intermediate cross-attention changes both state streams and adds dependency costs.
Illustrative example: Layer two can already read features and change later behavior modeling.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why exchange information between Wukong and HSTU at each layer?
WHALE: A Scalable Unified Model for Recommendation with Wukong-HSTU Architecture
Show answer and example
Answer: Sparse high-order fields and behavior sequences co-evolve instead of meeting only at final concatenation. Coupling may also cause interference.
Illustrative example: Attributes affect sequence interest and sequence states contextualize feature interactions.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why learn mean and variance, emphasize conflict in training, and serve only means?
Show answer and example
Answer: Gaussian score comparisons express proxy disagreement. Train using analytic wins, weight uncertain pairs, and regularize variance. Mean-only serving keeps cost low. Modeled uncertainty is not measured satisfaction.
Illustrative example: A click-friendly and a long-watch video create an uncertain pair; train on it but serve one mean score.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Does UAME sample Gaussian noise online, and how interpret local negatives?
Show answer and example
Answer: Serving uses mean only. Proxy losses do not reproduce author gains or refute private tests. Handle variance and zero-range weighting explicitly.
Illustrative example: Equal uncertainties give a zero min–max denominator; do not produce NaNs.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
How does uncertainty weighting differ from averaging pxtr scores?
Show answer and example
Answer: It emphasizes conflicting pairs and models comparison probability, unlike a fixed average. Variance is not observed satisfaction, and larger weights do not solve every conflict.
Illustrative example: Two candidates strong on different metrics should not be collapsed without showing uncertainty.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why predict a mean and variance in UAME?
Show answer and example
Answer: Multiple proxy scores can conflict. A Gaussian pair probability uses uncertainty during training, while serving reads only the mean.
Illustrative example: High clicks and low watch can make relative preference less certain.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why retain residual Semantic ID levels and add engagement and multi-view supervision in Pin-SCALE?
Show answer and example
Answer: Content codes transfer to new items but are not necessarily discriminative for behavior. Residual hierarchy and cascading pooling preserve coarse-to-fine details, while engagement, query–item, and item–item objectives align codes to recommendation. SIDs participate in training, not just replace vectors with integers.
Illustrative example: Illustration: Two living-room posts share a coarse theme but need finer audience distinctions; engagement and co-occurrence can refine them.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
Why doesn't one sample's positive layer weight wi necessarily change Pin-SCALE nearest-code argmin?
Show answer and example
Answer: Multiplying all distances by the same positive scalar leaves their order unchanged. The weight may instead change codebook updates or global learning. Check where it is applied; zero and candidate-dependent weights are different.
Illustrative example: Illustration: Multiplying distances two and five by three still selects two. Later assignment can change if weighted learning moves centers.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
How does engagement-aware tokenization differ from pure content reconstruction?
Show answer and example
Answer: Reconstruction preserves content geometry; engagement emphasizes differences affecting consumption. It can introduce exposure and popularity bias, so check tail generalization. The local fresh cohort's small denominator prevents interpreting large relative changes as industrial gains.
Illustrative example: Illustration: Behavior can distinguish similar renovation images by user purpose but may also favor a popular style over new content.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why use hierarchical SIDs for discriminative dense retrieval in Pin-SCALE?
Show answer and example
Answer: Cascading pooling maps transferable coarse and fine semantic codes into a retrieval embedding, and engagement/multi-view losses align it to the task. Discriminative nearest neighbors do not require autoregressive code decoding.
Illustrative example: Illustration: Pool gardening and indoor-plant codes into a two-tower vector instead of generating two codes at serving time.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Core mechanism
Why add teacher weights only to positive advantages?
Show answer and example
Answer: Verified positive samples already have outcome support; teacher ratio emphasizes useful tokens. Keep nonpositive learning unchanged. Clip and geometric-normalize weights without treating teacher preference as independent truth.
Illustrative example: Boost key steps in success; do not weaken negative updates just for teacher-approved wording.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Implementation details
Why normalize clipped ratios by sequence geometric mean?
Show answer and example
Answer: Bound ratio scale and make weights relative within a valid-token sequence. Exclude padding and keep nonpositive branch one. Local mini-suite doesn't reproduce full generation/training.
Illustrative example: Including padding changes every weight even if its loss is masked.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Method comparison
How does positive-only gating differ from weighting both signs?
Show answer and example
Answer: Nonpositive advantages keep weight one, preserving failure penalties. Two-sided weighting could weaken some negative feedback. The teacher does not relabel outcomes.
Illustrative example: Teacher preference for wording in a failed reply doesn't reduce failure penalty.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Basics
How does Distilled RL use teacher distributions inside RL rather than standalone imitation?
Show answer and example
Answer: Teacher/old-student ratios reweight task advantages at token level for positives only. Task outcome picks reinforcement; teacher focuses credit.
Illustrative example: A verified solution's teacher-supported key step can receive extra weight.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Core mechanism
How do composable Block, Residual, and Repeat help isolate ablations?
Show answer and example
Answer: Separate ordinary PyTorch modules from wiring such as repetition, residuals, and parallel branches. This reduces coupling to the trainer and distributed execution and aids matched comparisons. Composability is an engineering property, not quality evidence.
Illustrative example: Change serial to parallel wiring without changing data and training loop.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Implementation details
How avoid incorrect Residual or Parallel wiring?
Show answer and example
Answer: Residual(f)(x)=x+f(x); Parallel(f,g)(x)=f(x)+g(x). Check dimensions and repetition parameter sharing. Parallel addition is not serial composition, and not all upstream presets are local.
Illustrative example: f(x)+g(x) generally differs from g(f(x)) at the same shape.
Sections in the Chinese source: 核心公式, 复现边界. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Method comparison
Do DSL equivalence tests establish a better architecture?
Show answer and example
Answer: No. Equivalence checks preserve semantics under rewiring; architecture studies test quality after changes. Near-identical PPL supports equivalence, not improvement.
Illustrative example: Rewriting x+f(x) as Residual(f) should give the same result.
Sections in the Chinese source: 核心公式, 本地复现. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Basics
Why keep components as ordinary PyTorch modules?
Show answer and example
Answer: Tight coupling to training and distribution obscures structure and ablations. Plain modules and composable wiring improve transparency and portability across notebooks, CPUs, and GPUs.
Illustrative example: Replace one block without also replacing the training framework.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why change memory, item interface, and reasoning representation together in RecGPT-V3?
Show answer and example
Answer: CoT compression alone leaves repeated history reads and losses mapping natural language to items. Memory Hub updates traceable units incrementally, SID connects to products, and latent tokens cut reasoning output. They address different costs, while production ranking feedback also contributes.
Illustrative example: Illustration: Update only the photography memory after a camera view, emit an item code, and use shorter latent reasoning. Text length alone misses the other savings.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Why report RecGPT-V3 selection history and head share with final local metrics?
Show answer and example
Answer: Teacher warm-up and freezing were selected through iterations; the final single seed is not preregistered stability evidence. Rising head share can signal a popularity shortcut. Counts of memory and latent tokens are structural, not latency or 14B measurements.
Illustrative example: Illustration: Reducing 40 events to 14 inputs changes budget but does not promise a 65% speedup.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
What training protection does latent internalization add beyond deleting explicit CoT?
Show answer and example
Answer: Segment, cross-segment, and full-chain reconstruction transfer teacher information into latent slots. Removing output alone only saves text. Reconstruction does not prove faithful reasoning, and teacher quality and freezing matter. Local synchronous drift failed before teacher warm-up and freezing improved results.
Illustrative example: Illustration: Teaching four slots to support prior explanations differs from not printing an explanation; moving teacher and student together can destabilize targets.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Why store original behavior references and timestamps in a RecGPT-V3 memory unit?
Show answer and example
Answer: Provenance enables evidence tracing and incremental updates instead of letting summaries drift. Only relevant units change, but traceability is not lossless or causal correctness. Refresh and expiry still matter.
Illustrative example: Illustration: Link 'planning to camp' to dated browsing records so the right unit can be updated when needs change.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why separate RECAP LLM semantic edits and deterministic lifecycle management, then train from recommendation feedback?
RECAP: Feedback-Driven Streaming Semantic User Profiles for Short-Video Recommendation
Show answer and example
Answer: The LLM interprets new behavior but cannot guarantee capacity, merging, or eviction. A state machine enforces lifecycle while the LLM edits semantics. An evaluator rewards usefulness for future behavior instead of summary resemblance. Proxy bias can still be amplified.
Illustrative example: Illustration: Update running to cycling after new behavior and let rules evict stale entries; reward better prediction of later cycling.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
Who should enforce RECAP capacity |Pt|≤C, and why isn't a prompt sufficient?
RECAP: Feedback-Driven Streaming Semantic User Profiles for Short-Video Recommendation
Show answer and example
Answer: A deterministic state machine enforces lifecycle, merges, and evictions after validating LLM output. Bind updates, evidence, and rewards to the same round and exclude future targets. Expose invalid format and overflow rather than silently masking failure.
Illustrative example: Illustration: If seven entries arrive for capacity four, reject or merge by rule and log it; silent truncation does not prove compliance.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
Why can closed-loop RECAP GRPO increase reward while test ranking falls relative to open-loop SFT?
RECAP: Feedback-Driven Streaming Semantic User Profiles for Short-Video Recommendation
Show answer and example
Answer: SFT imitates updates, whereas GRPO optimizes evaluator preferences. If the proxy diverges from true next-item behavior, optimizing it can hurt generalization. This happened locally. Use independent evaluation plus reference, format, and diversity constraints.
Illustrative example: Illustration: A judge that likes detailed interests can reward verbosity while the user's main need gets diluted.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Are RECAP's LLM judge and two-tower evaluator the same role?
RECAP: Feedback-Driven Streaming Semantic User Profiles for Short-Video Recommendation
Show answer and example
Answer: No. The judge screens behavior pairs for label consistency to create training data. The two-tower evaluator learns profile–behavior matching and provides update rewards. Either can be wrong and neither is true user satisfaction.
Illustrative example: Illustration: A judge can approve a pair, but saying 'good' does not train a deployable two-tower reward model.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why include candidate-set statistics in real-estate LLM reranking rather than score each listing independently?
LLM-Based Re-Ranking for Real Estate Search
Show answer and example
Answer: Conversation needs can trade off budget, commute, and space across structured data and text. Set statistics provide current alternatives for relative decisions beyond vector similarity. List changes can invalidate scores, and reranking cannot recover missing listings.
Illustrative example: Illustration: If none is both quiet and near transit, the set helps identify the best compromise instead of matching each sales description alone.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Why recompute Agg(X) whenever a real-estate candidate set changes?
LLM-Based Re-Ranking for Real Estate Search
Show answer and example
Answer: It summarizes the current pool. Adding or removing listings changes relative budget and feasible options. Align structured fields and descriptions by identity and never choose candidates by real transaction labels. A local pointwise scorer is not full conversational reranking.
Illustrative example: Illustration: An initially cheap listing may cease to look cheap after adding lower-priced homes. The old mean misrepresents context.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
Why can't LLM relative reranking replace vector retrieval, or vice versa?
LLM-Based Re-Ranking for Real Estate Search
Show answer and example
Answer: Retrieval cheaply builds a pool from a large catalog but can miss complex needs. LLM reranking compares a small pool in detail but cannot choose absent items. Set conditioning adds cost; local movies lack real dialogue and housing constraints.
Illustrative example: Illustration: If the best school-and-commute match misses the top 100, reranking explanations cannot restore it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why combine structured property attributes with conversational needs instead of just listing prose?
LLM-Based Re-Ranking for Real Estate Search
Show answer and example
Answer: Descriptions can be vague; structured price, size, and commute specify constraints, while dialogue gives priorities and trade-offs. Set statistics add context. Language reasoning cannot replace factual attribute validation or rewrite facts.
Illustrative example: Illustration: Both listings say convenient; five versus twenty minutes to transit resolves a ten-minute requirement.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Core mechanism
Why can loop KV be compressed even when looped Transformer weights are shared?
Looped Latent Attention: Cross-Loop KV Compression for Looped Transformers
Show answer and example
Answer: Loops with shared weights still generate different KV. LLA stores shared low-rank latents and reconstructs loop-specific KV. Rank and error bound savings, and reconstruction costs compute.
Illustrative example: Four iterations can store a small latent plus four maps rather than four full KV copies, subject to reconstruction quality.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Implementation details
What must stay loop-specific when reconstructing KV?
Looped Latent Attention: Cross-Loop KV Compression for Looped Transformers
Show answer and example
Answer: A shared projection produces z, then separate loop matrices U reconstruct K/V. One identical KV for all loops removes the defining difference. The small model is not a post-hoc codec or fused server kernel.
Illustrative example: Use U¹ and U² on the same z to produce two K outputs.
Sections in the Chinese source: 核心公式, 复现边界. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Method comparison
What differs between LLA KV compression and reducing parameter count?
Looped Latent Attention: Cross-Loop KV Compression for Looped Transformers
Show answer and example
Answer: Weight sharing reduces parameter storage. Latent compression reduces history cache and adds reconstruction. Their memory scales differ.
Illustrative example: Cache keeps growing with generated length while model weights remain fixed.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Basics
How does sharing weights differ from sharing runtime state?
Looped Latent Attention: Cross-Loop KV Compression for Looped Transformers
Show answer and example
Answer: The repeated transform is the same, but KV depends on each loop's hidden inputs. LLA shares a latent while rebuilding each loop's K/V.
Illustrative example: One machine processing different intermediate materials produces different outputs.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
Why distill probability differences with and without hindsight skills on original actions instead of copying new expert traces?
Show answer and example
Answer: Skills come from completed-trajectory reflection. Compare conditional probabilities on states and actions actually visited by the student for denser local supervision. Hindsight explains original behavior without requiring a teacher to take another route. Deployment must not receive future answers for the same task, and skill mistakes can contaminate signals, requiring verification boundaries.
Illustrative example: If the student checked a directory and reflection finds it useful, compare the probability of that same action with and without the skill rather than substituting a teacher's command.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
What alignment must remain fixed when calculating a SEED skill log-probability difference?
Show answer and example
Answer: Use the same s_t, a_t, and policy evaluation convention, changing only skill conditioning z. Otherwise the gap includes state or action changes. Hindsight is training feedback; do not leak test-answer-derived skills into the policy.
Illustrative example: Scoring A with the skill and B without it does not measure the effect of skill guidance on A.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
How does SEED rescoring on original actions differ from letting a teacher generate a full successful trace?
Show answer and example
Answer: Rescoring keeps the actual student states and actions, measuring skill-conditioning changes. A new expert trace may visit different states. This is on-policy rescoring, not imitation of a separate ideal execution.
Illustrative example: If the student executed A, compare A in both conditions instead of replacing it with unexecuted B.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why does SEED turn reflections on completed trajectories into reusable hindsight skills?
Show answer and example
Answer: Reflections attached only to an answer do not accumulate as stable training signal. SEED extracts skills, compares probabilities for original actions with and without them, and combines dense distillation with outcome RL to reuse experience.
Illustrative example: A failure can yield the skill 'check tool preconditions first' as training guidance for later tasks.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why combine field-specific Q/K/V, noise gates, and layer supervision?
TMallGS: Scaling Unified Feature and Sequence Modeling for Generative E-commerce Search
Show answer and example
Answer: Separate projections address heterogeneous distributions, saliency and gates control noisy interactions, FiLM injects content, and intermediate loss guides deep learning. Gains cannot be attributed just to more heads.
Illustrative example: Downweight an accidental click while preserving an explicit search term and train intermediate relevance.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Does a very large relative NDCG gain establish a stable win?
TMallGS: Scaling Unified Feature and Sequence Modeling for Generative E-commerce Search
Show answer and example
Answer: No. Absolute hits are few and one seed with a low denominator inflates percentages. The run exercises field, gate, FiLM, bias, and intermediate training paths.
Illustrative example: A tiny absolute improvement can have a huge percentage.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does FiLM differ from concatenating content features?
TMallGS: Scaling Unified Feature and Sequence Modeling for Generative E-commerce Search
Show answer and example
Answer: FiLM generates per-coordinate scale and bias; concatenation passes features onward. Progressive supervision also assigns intermediate targets, not just one MLP.
Illustrative example: Content can emphasize hidden directions rather than merely add columns.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why learn field-specific Q/K/V in TMallGS?
TMallGS: Scaling Unified Feature and Sequence Modeling for Generative E-commerce Search
Show answer and example
Answer: Queries, user history, and item fields have different semantics and noise. Field projections, saliency, and gates shape interactions by reliability.
Illustrative example: A search term and years of browsing need not use one projection.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why keep recent events in a lightweight online Transformer when a long-history cache already exists?
Long-History User Transformers for Real-Time Ad Ranking
Show answer and example
Answer: The cache covers stable patterns but refresh lag excludes recent events. A short online window plus cached state responds without re-encoding all history. Auxiliary objectives support both representations. A small cache can lose information; do not mix fresh offline states with stale online caches in evaluation.
Illustrative example: Illustration: Overnight history records photography, but today's sudden childcare searches need a recent path without waiting for a refresh or re-encoding all past events.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
How should the local 24-step offline, four-step online model enforce input cutoffs?
Long-History User Transformers for Real-Time Ad Ranking
Show answer and example
Answer: Both branches use only events before the prediction. Define overlap handling and never build a cache from the full sequence including test events. Next-item and feedback auxiliary labels stay on the training side. A 32-dimensional cache is a scale-down, not the original history length or feature scale.
Illustrative example: Illustration: A cache containing the final film leaks it even if the online branch reads only four earlier events.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
What costs move to the background in a two-stage async model compared with request-time full history encoding?
Long-History User Transformers for Real-Time Ad Ranking
Show answer and example
Answer: The background encodes long history and caches fixed-width state, avoiding repeated long self-attention at serving time. Refresh, storage, invalidation, and compression costs remain. The source lacks actual Yandex feature-store measurements, so structure alone cannot establish latency gains.
Illustrative example: Illustration: Requests can share a daily summary, but if updates are frequent and requests rare, background compute may not pay off.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why pair the offline Long-History Transformer cache with a recent online path?
Long-History User Transformers for Real-Time Ad Ranking
Show answer and example
Answer: Async long history retains stable interests but lags. A small online model reads recent changes and the cache, combining coverage with responsiveness. Compression can lose details, and auxiliary pretraining and ad fine-tuning are distinct stages. Concatenating a long vector does not reproduce all of this.
Illustrative example: Illustration: Baby-product browsing can change a prediction immediately even while the background cache still emphasizes photography.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why screen downstream rewards for future engagement instead of adding all immediate behavior to ranking?
Long-term User Engagement Optimization through Model-agnostic Downstream Rewards Learning
Show answer and example
Answer: Immediate responses can conflict with satisfaction or amplify noise. Test candidate signals for future engagement and connect only selected heads. Predictive association does not imply causal benefit; evaluate each product and do not add short-term gains as retention.
Illustrative example: Opening a large image may be accidental; deeper collection browsing might predict later activity better and is worth checking.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Where does future-window information belong when training a downstream reward head?
Long-term User Engagement Optimization through Model-agnostic Downstream Rewards Learning
Show answer and example
Answer: Use it for reward screening and supervision, not features beyond request time. Select fusion strength on validation and freeze it. Local future category coverage is a proxy, not platform retention; do not read held-out future events.
Illustrative example: Tomorrow's new-topic exploration can label today's prediction, but cannot be today's input.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
How does adding a downstream reward head differ from replacing the ranker?
Long-term User Engagement Optimization through Model-agnostic Downstream Rewards Learning
Show answer and example
Answer: An auxiliary head preserves the backbone and adds a filtered long-horizon proxy to loss or score fusion across surfaces. Replacement changes more features and serving logic. Model-agnostic integration does not imply shared label semantics or improved component metrics.
Illustrative example: A feed can keep relevance and add deep-consumption prediction. Notification weights can differ, and saves are not notification retention.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why screen immediate behaviors with future engagement in Downstream Rewards?
Long-term User Engagement Optimization through Model-agnostic Downstream Rewards Learning
Show answer and example
Answer: Short-term clicks or saves can be fleeting. Find signals predictive of later activity, then train online-predictable auxiliary heads to account for longer-horizon value. Association remains provisional and each product needs independent verification.
Illustrative example: Compare accidental image opens with deeper collection exploration before selecting a reward target.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why not always choose the ad count maximizing immediate ad revenue?
Adaptive Ad Load Design for Sponsored Search Markets: Evidence, Theory, and Deployment
Show answer and example
Answer: Count affects organic experience and conversions as well as revenue. Randomized field experiments estimate count responses; constrained decisions use Lagrangian trade-offs. Correlation logs or ranking proxies are not causal or market evidence.
Illustrative example: Shopping and navigation queries can react differently to adding one ad.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Why enumerate feasible counts with consistent revenue and conversion definitions?
Adaptive Ad Load Design for Sponsored Search Markets: Evidence, Theory, and Deployment
Show answer and example
Answer: Score each legal k under inventory and business limits, using comparable revenue and weighted conversion terms. Tune coefficients on validation, not test.
Illustrative example: Do not output five ads when zero to three are legal or compare inconsistent per-request/per-ad metrics.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
Why use randomized field evidence instead of comparing high-revenue logs?
Adaptive Ad Load Design for Sponsored Search Markets: Evidence, Theory, and Deployment
Show answer and example
Answer: Existing policies choose load based on intent, which confounds revenue differences. Randomized load helps identify intervention effects. Public ranking without interventions cannot establish causal count effects.
Illustrative example: Many ads on already high-intent requests does not justify increasing all loads.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why optimize per-request ad quantity, not just order?
Adaptive Ad Load Design for Sponsored Search Markets: Evidence, Theory, and Deployment
Show answer and example
Answer: Ranking fills given slots; load chooses their number and affects experience, revenue, and conversion. Estimate marginal response over feasible counts.
Illustrative example: Identical ad candidates can support different loads for shopping and exploratory requests.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why must the memory controller count UCB visits per discrete state rather than use global action frequencies?
Memory as a Controlled Process: Learned Adaptive Memory Management for LLM Agents
Show answer and example
Answer: An operation useful early may not help later or under memory pressure. State-local values and counts let the controller learn those trade-offs separately. It explores unvisited actions first, then updates from terminal success and efficiency. No extra LLM call applies to selection; retrieval and execution still have costs.
Illustrative example: Example: Deep retrieval may help at the start and delay completion near the end. Count attempts separately instead of sharing one global success rate.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
Can MemCon's Q update use sample-mean UCB or TD bootstrapping?
Memory as a Controlled Process: Learned Adaptive Memory Management for LLM Agents
Show answer and example
Answer: No. The local implementation follows v3 state-local counts, discounts terminal success and efficiency by distance, and updates repeated state-action pairs in reverse order. It does not bootstrap future Q. Unvisited actions go first with initial Q tie-breaking. Differences from mutable upstream main counts are documented.
Illustrative example: Example: An action seen twice in one episode receives two discounted updates in reverse order, not one reward averaged by visits.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
What is the trade-off between tabular UCB and asking an LLM to choose a memory policy every round?
Memory as a Controlled Process: Learned Adaptive Memory Management for LLM Agents
Show answer and example
Answer: Tabular control needs no extra model call and has interpretable states and actions, but limited discrete features cannot express arbitrary semantic contexts. LLM control is flexible but costs calls. State-local UCB does not guarantee convergence as the environment and memory change.
Illustrative example: Example: A repetition flag can trigger deeper retrieval immediately. Semantically different tasks with the same discrete state may still look identical to the table.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Why does MemCon make whether to use memory a control action?
Memory as a Controlled Process: Learned Adaptive Memory Management for LLM Agents
Show answer and example
Answer: Retrieval, plan injection, query changes, and maintenance cost work. Some simple tasks are cheaper without memory. MemCon selects based on discrete progress, repetition, and memory size, then learns from terminal success and efficiency. Memory need not run on every turn.
Illustrative example: Example: With an obvious next tool, skip retrieval. After repeated failure and with a relevant successful plan, inject it or search deeper.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
If SlimPer already selects evidence with attention, why explicitly dot-product it with knowledge templates?
SlimPer: Make Personalization Model Slim and Smart
Show answer and example
Answer: Select builds queries from the current bank and aggregates a few user evidence vectors. Match compares those vectors with templates from the same bank, yielding a query-by-template relevance matrix. Refine updates knowledge from these explicit relations rather than treating the aggregate as final interest. Retrieval and aspect-specific relevance are separate.
Illustrative example: For running shoes, Select finds running distances and sizes. Match evaluates comfort and intended use separately. Finding evidence does not tell which candidate aspect it supports.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
How does SlimPer keep its knowledge base fixed in size while reading the full history at every layer?
SlimPer: Make Personalization Model Slim and Smart
Show answer and example
Answer: Only a small set of candidate-specific slots is updated. Original user tokens are revisited as read-only evidence. Select retrieves relevant evidence, Match compares it with templates, and Refine updates slots. The full hidden history does not propagate between layers. Candidates share user encodings, but each evolves its own knowledge base rather than using one final interest vector.
Illustrative example: Shoe and camera candidates can read the same purchase records but organize size and photography signals in separate banks. Slots updated for shoes must not be reused unchanged for cameras.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Once SlimPer fixes the knowledge-base size, can later layers stop reading long histories?
SlimPer: Make Personalization Model Slim and Smart
Show answer and example
Answer: No. Each layer queries shared original tokens. It saves updates to N history states at each layer but still accesses KN query–history pairs. Short MovieLens runs, AdamW, and analytical element counts do not reproduce production QPS or Shampoo results.
Illustrative example: A small bank must still revisit the evidence; it is not a one-shot compression that discards the history.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Does not updating all history at each SlimPer layer eliminate relationships between adjacent behaviors?
SlimPer: Make Personalization Model Slim and Smart
Show answer and example
Answer: No. Section 2.5 adds local context when tokenizing events, for example via convolution or windowed attention. Later layers repeatedly read those tokens and update a fixed bank. This avoids deep all-pairs history interaction, not every historical relation. Fixed slots cap intermediate state; storing and reading inputs still grows with history length.
Illustrative example: Search, compare, and add-to-cart events can get local context before candidate-specific slots read them, without rewriting the full history at every layer.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
Why is SlimPer not just a sequence encoder when compared with HSTU or LONGER?
SlimPer: Make Personalization Model Slim and Smart
Show answer and example
Answer: SlimPer models user–item relevance. Its knowledge base also receives item and cross features, and each Select–Match–Refine layer forms a personalized ranking state. A matched comparison needs candidate interactions and the task head, not only a user-sequence vector.
Illustrative example: Two candidates can produce different knowledge bases from the same history. Do not compare them solely to a candidate-independent encoder output.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why suppress related history after buying and restore it near replenishment?
Show answer and example
Answer: A purchase can complete an intent. Find its related history and estimate personal rhythm; time gates and log masks suppress then restore it. Completion is not dislike. Local ratings proxy purchases, not actual replenishment.
Illustrative example: Reduce detergent repeats immediately, then restore after likely use rather than suppress forever.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Where is SAM's mask applied, and what are the local limits?
Show answer and example
Answer: Add a positive log mask to attention logits with numeric safeguards, not a multiplier on final scores. No real replenishment or censored purchase logs; only mechanisms and failure boundaries.
Illustrative example: Guard near-zero masks without claiming accurate repurchase cycles.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does SAM differ from a fixed post-purchase cooldown?
Show answer and example
Answer: It personalizes suppression by history and long-term purchase rhythm. Errors can still hurt, and nonreplenishable products may not fit.
Illustrative example: Coffee and televisions should not share a one-week rule.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Does SAM blacklist same-category items after a purchase?
Show answer and example
Answer: No. It softly suppresses attention to related history and restores it over time, not permanent candidate filtering. Other evidence can still support recommendations.
Illustrative example: Buying shampoo reduces old browsing influence without banning hair products.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why split dense regions and merge sparse regions in Airbnb proximity keys?
Proximity Features: Privacy-Compliant Cold-Start Personalization at Airbnb
Show answer and example
Answer: A fixed grid can be too coarse downtown and contain too few users elsewhere. Hash subdivision and coarser merging maintain group support. Daily aggregates provide anonymous cold-start priors, not persistent personal identity. Geography proxies cohort preferences, not individual interests.
Illustrative example: Illustration: Split a block with thousands of users, but enlarge an equally sized suburban cell with ten people.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
Why version proximity coordinate quantization, hashes, and daily aggregation?
Proximity Features: Privacy-Compliant Cold-Start Personalization at Airbnb
Show answer and example
Answer: Group behavior is accumulated by stable keys. Changing precision or hashing can move a region to a new bucket and mismatch serving history. Handle small groups and missing coordinates explicitly, without secretly using precise personal locations. Small MovieLens buckets are not production groups of around a thousand or privacy evidence.
Illustrative example: Illustration: Changing decimal precision today and looking up yesterday's key can read the wrong aggregate; successful base64 encoding proves little.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
What does Airbnb proximity aggregation add beyond a global cold-start aggregate?
Proximity Features: Privacy-Compliant Cold-Start Personalization at Airbnb
Show answer and example
Answer: Global aggregates give all anonymous requests one prior. Proximity aggregates condition on nearby groups with adequate support. They capture local demand but reduce samples per bucket and may amplify regional bias. ZIP proxies test local signals, not recovered personal tastes.
Illustrative example: Illustration: A group may favor weekend trips while the global population favors large cities; one traveler in that group can still want an international trip.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why are Proximity Features group keys rather than anonymous personal IDs?
Proximity Features: Privacy-Compliant Cold-Start Personalization at Airbnb
Show answer and example
Answer: Keys aggregate geographic groups meeting count thresholds, with denser regions split and sparser ones merged. Serving consumes group behavior without persistent individual tracking. Cohort priors miss fine individual preferences, and proxy ZIP counts are not production privacy thresholds.
Illustrative example: Illustration: Nearby users' short-stay preferences can seed an anonymous request without identifying one user's bookings.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Which two kinds of locality do TCL and TDL address in NONTP next-token prediction?
Not Only NTP: Extending Training Signal Coverage for Generative Recommendation
Show answer and example
Answer: TCL predicts a later EMA-teacher state, extending immediate next-step supervision. TDL predicts the target through a shared item head from earlier cross-domain states, helping sparse domains. Both are training-only; inference stays on the original backbone and cannot read future states.
Illustrative example: Illustration: Before a lamp purchase, TCL teaches longer trajectories and TDL uses preceding home-content behavior. Serving sees only events already observed.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why must TDL use pre-target cross-domain history and the same item head?
Not Only NTP: Extending Training Signal Coverage for Generative Recommendation
Show answer and example
Answer: Time order prevents leakage, domain mismatch supplies cross-business transfer, and averaging into the shared NTP head trains the target item space directly. Skip empty groups rather than inventing labels; an independent head is not the same transfer interface.
Illustrative example: Illustration: Predict a shopping target from earlier other-domain states, not videos afterward. No cross-domain history means no TDL evidence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How is TCL's EMA teacher more stable than a direct online future-state target?
Not Only NTP: Extending Training Signal Coverage for Generative Recommendation
Show answer and example
Answer: EMA smooths updates and gives a stopped-gradient target rather than two sides chasing each other. Future state remains training-only; use other users as negatives rather than mislabeling nearby same-user interests. Stability does not ensure correctness and can preserve old biases.
Illustrative example: Illustration: A slow teacher changes less than a recent online update, but a consistently popularity-biased teacher is still biased.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why add NONTP auxiliary modules only during training?
Not Only NTP: Extending Training Signal Coverage for Generative Recommendation
Show answer and example
Answer: TCL and TDL teach long-range and cross-domain signals to the backbone without future teachers or extra branches at serving time. Richer supervision is not uniformly better, especially with weak domain labels and short histories. Future targets are not available online.
Illustrative example: Illustration: Train for a trajectory several steps ahead but serve with the current history and original item head.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Core mechanism
Why scale user, item, and context sub-towers separately in MESH before global interaction?
MESH: Scaling Up Retrieval with Heterogeneous Content Unification
Show answer and example
Answer: Flattening early lets strong common-item gradients obscure fresh and tail signals. Residual cross interactions within each sub-tower retain specialized capacity; a context gate then adjusts user–item affinity per request. A cacheable item tower does not remove candidate-dependent scoring costs.
Illustrative example: Illustration: Model a tail item's content before it competes with popular exposure counts, then judge the item in the request context.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Implementation details
Why does RGBC gate elementwise user–item affinity with context and add a context residual?
MESH: Scaling Up Retrieval with Heterogeneous Content Unification
Show answer and example
Answer: The product models corresponding dimensions of user and item vectors. A sigmoid context gate selects affinities useful now, while the residual retains context itself. Align dimensions and identity across towers. Averaging context loses sample conditioning, and layer-normalization and residual placement matter.
Illustrative example: Illustration: Commuting and weekend contexts can emphasize different affinity dimensions for the same pair, while context independently represents time. A final constant bias cannot do this.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Method comparison
When can MESH's parallel towers be faster than a flat serial model?
MESH: Scaling Up Retrieval with Heterogeneous Content Unification
Show answer and example
Answer: Independent towers permit concurrency and async item caching, but speed depends on runtime, hardware, batch size, and synchronization. A parallelizable graph is not necessarily executed in parallel by small-batch Python. The local slowdown is valid evidence. Compare quality and compute separately instead of inferring throughput from tower count.
Illustrative example: Illustration: Three towers run sequentially in one CPU loop with no GPU concurrency and more scheduling overhead; they do not become three times faster by design alone.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Basics
Why scale inside the user, item, and context sub-towers first in MESH?
MESH: Scaling Up Retrieval with Heterogeneous Content Unification
Show answer and example
Answer: Flat interaction can let frequent evergreen signals dominate gradients. Separate cross interactions retain structure before a context gate adjusts affinity. The aim is more balanced capacity use, not guaranteed fresh/tail gains; inspect cohort and aggregate metrics.
Illustrative example: Illustration: Encode stable preferences, item freshness, and request context separately so popular-item statistics do not drown out context from the start.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Core mechanism
Why does DANet separate high- and low-frequency discount history rather than using only today's discount?
Show answer and example
Answer: Current price cannot describe long-term sensitivity and short-term promotion reactions. DPN separates stable patterns from fast variations and modulates preferences with similar purchases and promotion context; another network models general interest. Purchase-only discount labels are auxiliary and do not identify causal effects.
Illustrative example: Two users see 20% off, but one shops only during annual events and the other reacts to regular small offers.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Implementation details
Why does DANet regress observed discounts only on purchases?
Show answer and example
Answer: The target is a transaction discount conditioned on purchase, which unpurchased items lack. Mask with purchase positives rather than inventing labels. After FFT filtering, low-frequency reconstruction plus high-frequency residual should recover the sequence. Local popularity/rating intensity is not real coupon-price regression.
Illustrative example: An unpurchased shoe has no transaction discount; do not assign it zero. Use actual purchase discounts for this head and separate click or ranking supervision.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Method comparison
What structure does DANet's FFT frequency decomposition preserve beyond a recent discount average?
Show answer and example
Answer: A mean compresses trends and fluctuations into one number. Frequency separation reveals promotion rhythm, but depends on temporal sampling and quality, and high frequencies can be noise. Public data without prices provide proxy diagnostics only.
Illustrative example: Steady 10% off with a rare 50% sale can average similarly to weekly swings between 20% off and full price, but behavior can differ.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Basics
Why model general interest and discount preference separately in DANet?
Show answer and example
Answer: Category interest is not the same as willingness to wait for discounts. IPN models general interest, DPN learns discount rhythms, and user and promotion context modulate them. The signals can interact but are not interchangeable.
Illustrative example: Two sneaker fans may buy at full price or wait for a season-end sale despite similar browsing.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Core mechanism
Why decide whether to trigger causal retrieval by uplift instead of underlying shopping probability?
Deep-learning Causal Retrieval Optimization for Efficient e-commerce Distribution in Pinterest
Show answer and example
Answer: High-propensity users might buy without retrieval, so allocate compute by the outcome changed by triggering. Use randomized logs to learn treated and control outcomes, doubly robust correction to estimate increment, and replay to select a threshold. Adequate experimental support and overlap are still necessary.
Illustrative example: A likely buyer may gain little, while a less likely buyer can be persuaded by relevant candidates. Under a budget, compare intervention increments rather than base propensity.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Implementation details
Why check propensity values near zero or one in causal retrieval?
Deep-learning Causal Retrieval Optimization for Efficient e-commerce Distribution in Pinterest
Show answer and example
Answer: The DR residuals divide by e(x) or 1−e(x); extremes amplify noise, and missing overlap can prevent estimation. Choose trigger thresholds on independent replay or validation rather than test. Document any clipping because it changes estimator bias and variance.
Illustrative example: A subgroup triggered only 1% of the time can give one outcome enormous weight; a high score caused by that denominator is not high-confidence uplift.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Method comparison
What does DR uplift add beyond two outcome models for treatment and control?
Deep-learning Causal Retrieval Optimization for Efficient e-commerce Distribution in Pinterest
Show answer and example
Answer: The model difference estimates the outcome gap; a propensity-weighted residual uses observed assignment information to correct fitting error. Double robustness depends on its assumptions and suitable model specification, not immunity to arbitrary bias. Local synthetic randomized treatments do not establish production uplift.
Illustrative example: The residual may correct underestimated treated purchases, but cannot fix zero support when this group has never received treatment.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Basics
What are the treatment and outcome in causal retrieval?
Deep-learning Causal Retrieval Optimization for Efficient e-commerce Distribution in Pinterest
Show answer and example
Answer: Treatment is whether the shopping candidate generator runs; outcomes are later business responses across events. Estimate the trigger/no-trigger difference for comparable users, not ordinary shopping-model accuracy. Both decisions must be sufficiently represented in the data.
Illustrative example: The question is whether running shopping retrieval changes purchase or consumption for the same context, not whether that user likes to shop anyway.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Core mechanism
Why configure Prompt Generation's preprocessor, projector, and merger with templates?
Prompt Generation Technical Report
Show answer and example
Answer: Separate training and serving implementations drift. One protocol composes cleaning, projection, and fusion/compression with common interpretation. Merger changes token cost and information loss, not merely text assembly. Mean compression does not justify discarding all order.
Illustrative example: Illustration: Configure cleaning, projection, and pooling for one title token in both paths; truncating to ten words only on one side creates skew.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Implementation details
Why check PG_train(x,c)=PG_serve(x,c) by replaying raw serving features?
Prompt Generation Technical Report
Show answer and example
Answer: Matching JSON does not ensure tokenizer, dependencies, truncation, or missingness match. Replay inputs and compare final tensors with versioned interpreters/configurations. Validation selected Title but test regressed; shorter compression does not establish effectiveness.
Illustrative example: Illustration: If serving strips model numbers but training retains them, replay reveals skew despite the same configuration name.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Method comparison
What are the defaults and costs of parameterless mean merger versus learned compression?
Prompt Generation Technical Report
Show answer and example
Answer: Mean pooling is simple, cheap, and consistent without new weights. It loses order and details. Learned compression is flexible but adds cost and overfitting. Compression ratio is not relevance evidence, and mean need not fit every feature.
Illustrative example: Illustration: Pool redundant item attributes, but averaging a history with negation and ordering can lose critical meaning.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Basics
Why separate feature type from processing components in Prompt Generation?
Prompt Generation Technical Report
Show answer and example
Answer: Text, embedding, combo, and sequence say what the input is; preprocessing, projection, and merger say how to transform it. Orthogonal configuration composes without changing model code, but bad configuration can lose data or introduce skew.
Illustrative example: Illustration: A title and a behavior sequence can reuse component families with different settings; text does not imply one fixed pipeline.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why does next-POI generation need spatiotemporal context rather than ordinary place IDs?
Guess Where You Go: Generative Next Point-of-Interest Recommendation in Amap
Show answer and example
Answer: Distance, time, and travel state constrain feasibility. Hierarchical semantic IDs organize places, contextual features constrain present needs, and curriculum training adds more complex feedback gradually. Long-horizon rewards need actual observations; co-occurrence or one hit does not establish route-planning value.
Illustrative example: A coffee lover can reach different shops on a morning commute and a weekend trip.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Which POI SID training targets must not become request features?
Guess Where You Go: Generative Next Point-of-Interest Recommendation in Amap
Show answer and example
Answer: Target locations and future stay feedback can enter loss only. Inputs use prior trajectories and contemporaneous context, not next-stop coordinates for filtering. Codes map to the matching directory version, and proxy score fusion is not full policy training.
Illustrative example: Filtering by the true destination's shopping district leaks the answer; use the user's current area and legal constraints.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
What stages do curriculum pretraining/SFT and long-term POI policy updates address?
Guess Where You Go: Generative Next Point-of-Interest Recommendation in Amap
Show answer and example
Answer: Curriculum and supervision establish place semantics, timing, spatial relationships, and generation; later feedback adjusts long-term preference without exploring a huge space from sparse reward alone. Public movie proxies do not verify mobility or POI capability.
Illustrative example: Learn legal regions and visit sequences first, then refine routes based on satisfaction rather than expecting one reward to teach city geography.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Why does next-POI generation rely on spatiotemporal context more than content recommendation?
Guess Where You Go: Generative Next Point-of-Interest Recommendation in Amap
Show answer and example
Answer: Current area, time, accessibility, opening hours, and trip stage affect destinations. SIDs represent places; context selects meaningful paths. Pure co-occurrence can recommend semantically related but unreachable places.
Illustrative example: A lunchtime request in Shanghai should favor nearby food, not a favorite Beijing café.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Core mechanism
Why can ZoRRO rank without training, and what limits it?
ZoRRO: A Zero-Weight Personalized Recommender System for Scalable News Recommendation
Show answer and example
Answer: Combine history recency, semantic similarity, and category in fixed rules. New history updates scores immediately without fitting, but cannot learn complex collaborative and user-specific relations. Features, data, and hyperparameters still matter.
Illustrative example: Recent technology reading raises related articles without learning that one user prefers culture on weekends.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Implementation details
Does a movie proxy establish high-throughput online news behavior?
ZoRRO: A Zero-Weight Personalized Recommender System for Scalable News Recommendation
Show answer and example
Answer: No. MovieLens lacks news timeliness, private traffic, and serving. Fix age, cutoff, category source, and weights; do not tune per test case and claim zero-training generalization.
Illustrative example: Movie age is only a mechanism proxy for fresh news.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Method comparison
How does time-decayed semantic scoring differ from popularity?
ZoRRO: A Zero-Weight Personalized Recommender System for Scalable News Recommendation
Show answer and example
Answer: It changes by individual history and age, while popularity mostly shares global trends. Category adds structure; semantic similarity can still cause repetition and miss freshness.
Illustrative example: Two readers can rank the same articles differently.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Basics
Does zero training mean ZoRRO needs no representation model?
ZoRRO: A Zero-Weight Personalized Recommender System for Scalable News Recommendation
Show answer and example
Answer: No. It uses existing article embeddings and categories, with fixed temporal/semantic rules instead of fitting ranker parameters. Existing models and hyperparameters still cost resources.
Illustrative example: Retrieve with external vectors without backpropagating on news logs.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Core mechanism
Why quantile-match before fusing semantic and lexical scores in ELISE?
Multilingual Semantic Retrieval for Apple Music Search
Show answer and example
Answer: Raw scores differ in scale and distribution, so a wide-range scorer can dominate. Map each score to its relative distribution position, fuse under controls, and limit semantic result count. Percentiles are not click probabilities and do not replace exact-entity checks; language shifts can require recalibration.
Illustrative example: Lexical scores in the tens would swamp cosine scores near 0–1; percentile matching balances them while preserving exact song-title controls.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Implementation details
Why lock ELISE's mapping, fusion weights, and semantic quota before test?
Multilingual Semantic Retrieval for Apple Music Search
Show answer and example
Answer: Each changes candidates and ranks. Fit them on training or validation, freeze them for test, and use the same serving quota. Do not hide negative proxy results by selecting the best test mix.
Illustrative example: Choosing 70% semantic after examining test makes the report optimistic; lock the ratio and mapping before evaluating new requests.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Method comparison
How does quantile matching differ from min-max scaling?
Multilingual Semantic Retrieval for Apple Music Search
Show answer and example
Answer: Min-max primarily adjusts range; quantiles map positions in empirical distributions and can align different shapes. Equal percentiles are not equal relevance probabilities and drift changes their meaning. Equal numeric ranges do not establish probability calibration.
Illustrative example: A heavy lexical tail and narrow semantic distribution may not align by multiplying by a constant; novel queries can still violate old empirical distributions.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Basics
Why keep lexical and semantic retrieval in ELISE?
Multilingual Semantic Retrieval for Apple Music Search
Show answer and example
Answer: Lexical search handles exact entities, model IDs, and constraints; dual encoders handle paraphrase and semantic association. Calibrate scores and cap semantic results so broad matches do not override explicit requirements. Either path can still err.
Illustrative example: Search a specific camera model lexically, while semantic retrieval can find waterproof gear for rainy hikes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
Why isn't one PPO ratio interval the same policy distance at every old probability?
Show answer and example
Answer: Local geometry and KL cost depend on old probability. RIPO uses Fisher–Rao probability-dependent bounds, not different advantage labels. Actual update stability still needs verification.
Illustrative example: Doubling rare and common actions affects remaining mass differently.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
Should RIPO dynamic clipping read pre- or post-update probabilities?
Show answer and example
Answer: Set boundaries from frozen old-policy action probabilities while evaluating current ratios. Recomputing from current shifts the trust region during updates. Candidate tests aren't long-CoT training.
Illustrative example: Increasing an action's probability should not rewrite that round's original bound.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
How do adaptive RIPO bounds differ from raising global PPO ε?
Show answer and example
Answer: Global ε broadens every action. RIPO conditions bounds on old probability rather than uniformly becoming more aggressive, without guaranteeing universal stability.
Illustrative example: Exploration needs in low-probability regions need not relax frequent actions equally.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
Why does a fixed ratio clip give uneven distance budgets in RIPO?
Show answer and example
Answer: Equal ratio changes have different geometric effects at different old probabilities. Fisher–Rao geometry adjusts bounds for more comparable local budgets.
Illustrative example: The same relative increase for rare and frequent actions need not move the distribution equally.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Core mechanism
How does active reference sampling differ from a KL penalty?
Show answer and example
Answer: KL constrains drift on sampled states without guaranteeing useful reference behaviors appear. Frozen reference rollouts add that coverage. Anchors may be bad; tune mix and don't generalize candidate proxies.
Illustrative example: Active replay reintroduces the old verification route that a passive penalty may never sample.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Implementation details
What is local ARMOR reference, and are its samples on-policy?
Show answer and example
Answer: The initialized candidate policy; log anchors separately from current student samples. No paper long-horizon pool; mixing tests data flow, not stability.
Illustrative example: Don't label an old-policy action as generated by the current policy.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Method comparison
Are reference rollouts equivalent to adding reference KL?
Show answer and example
Answer: No. Rollouts change visited states; KL regularizes distributions on existing states. Anchors may be stale or wrong and need evaluation.
Illustrative example: A missing old solution isn't practiced merely because KL is present.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Basics
Why sample actively from reference in ARMOR?
Show answer and example
Answer: A passive penalty only acts on visited data and can't recover paths current policy never samples. Reference anchors restore actual training coverage.
Illustrative example: If student only takes a shortcut, old policy may still generate a verification route.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Core mechanism
Why reduce only positive high-tail-risk credit?
Show answer and example
Answer: Correct trajectories can contain unlikely, unsound steps. Reduce their positive reinforcement smoothly while retaining negative signals for failures. Contextual risk is not a low-probability equals wrong rule; no real local predictor.
Illustrative example: A lucky correct guess needn't be rewarded as strongly, but its negative feedback in a failed trajectory remains.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Implementation details
Does local TACO reproduce a contextual tail-risk predictor?
Show answer and example
Answer: No. Candidate actions verify credit direction, not long-CoT token risk. Smooth weights must be between zero and one; don't simply delete all unlikely tokens.
Illustrative example: A supplied risk label tests branching, not automatic detection of implausibility.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Method comparison
Does TACO scale positive and negative advantages the same way?
Show answer and example
Answer: No. Reduce positive credit only for high-risk tokens, leaving nonpositive advantages unchanged. This preserves failure learning.
Illustrative example: A rare word in a failed response should not escape the original penalty solely for high risk.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Basics
Why can a correct TACO response contain tokens that shouldn't be reinforced?
Show answer and example
Answer: Terminal correctness only validates the result. Uniform positive credit can reinforce lucky or implausible intermediate tokens. Tail risk smoothly reduces their positive contribution.
Illustrative example: A correct computation with a false explanation is not wholly exemplary.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
Why does normalizing KL by turn length change the training weight of long and short turns?
Show answer and example
Answer: Token sums naturally give long turns more loss, potentially drowning out short critical decisions. Turn-normalized supervision averages within a turn; separate probes allocate rollout depth to informative interactions under limited teacher budget. Weighting and sampling changes do not imply that deeper or greater supervision always helps.
Illustrative example: A key tool choice may be 10 tokens and an explanation 200. Turn averaging reduces the twentyfold weighting difference caused by length alone.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
After dynamic truncation, may TurnOPD retain teacher scores from omitted turns in the loss?
Show answer and example
Answer: Supervision must match actual cutoff L* and not use fictitious targets for uncollected turns. Respect valid response lengths. The current teacher is a deterministic task oracle, not a task-specific large model.
Illustrative example: If this rollout stops at turn three, prefabricated answers for turns four through six are not training evidence for it.
Sections in the Chinese source: 核心公式, 复现边界. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
How do token-averaged KL and a sum of length-normalized turn KL differ in weighting?
Show answer and example
Answer: Token-level aggregation gives long turns more terms. Dividing each turn by its own length gives turns more comparable weights. The same token batch can still define different objectives.
Illustrative example: A 100-token turn does not automatically get ten times the weight of a 10-token turn if both are first normalized separately.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why does TurnOPD study both rollout depth and within-turn supervision normalization?
Show answer and example
Answer: Later turns differ in training value and cost. Probes choose sampling depth, while turn-normalized KL changes supervision shares for long and short turns, separately controlling sampling budget and update weight.
Illustrative example: Sampling fewer turns and reducing the relative weight of one long turn are distinct savings.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
Why does SIS set accepted ratios to one, and does this make the whole trajectory on-policy?
Turning Off-Policy Tokens On-Policy: A Plug-in Approach for Improving LLM Alignment
Show answer and example
Answer: Under ideal rejection sampling the accepted conditional action distribution is corrected at a fixed prefix, so no original ratio is needed there. Rejects follow host correction. Whole histories remain old, and top-K is approximate.
Illustrative example: Accepting one position never sets all trajectory ratios to one.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
Which policy supplies SIS top-K, and what happens to rejections?
Turning Off-Policy Tokens On-Policy: A Plug-in Approach for Improving LLM Alignment
Show answer and example
Answer: Approximate the envelope on behavior top-K. Set accepted ratios to one, keep GRPO/DAPO/GSPO correction for rejected tokens, rather than deleting or resetting them.
Illustrative example: One rejected token does not force dropping the full reply.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
Is setting accepted token ratios to one the same as clipping to one?
Turning Off-Policy Tokens On-Policy: A Plug-in Approach for Improving LLM Alignment
Show answer and example
Answer: No. Acceptance is a distributional rejection-sampling event, not numerical capping. Exact envelopes and behavior top-K approximations have different guarantees.
Illustrative example: Replacing every large ratio with one gives no sampling guarantee.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
Why convert some old-policy SIS tokens into usable current-policy samples?
Turning Off-Policy Tokens On-Policy: A Plug-in Approach for Improving LLM Alignment
Show answer and example
Answer: Asynchrony/reuse induce staleness and products of ratios have high variance. Rejection sampling reduces correction for accepted tokens; the host method handles rejected ones without dropping all data.
Illustrative example: Retain useful accepted positions within an old batch.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Core mechanism
Why does Cluster GOOBS mostly draw negatives from the positive's semantic cluster but keep random negatives?
Real-Time Hard Negative Sampling via LLM-based Clustering for Large-Scale Two-Tower Retrieval
Show answer and example
Answer: Same-cluster items are closer to the decision boundary than easy random negatives; random draws preserve global coverage. The change is segmented maintenance and sampling in the live out-of-batch pool, not adding an online LLM. Same-cluster items can be false negatives, so validate the ratio.
Illustrative example: Other running shoes are harder negatives than refrigerators, but occasional cross-category draws preserve broader distinctions.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Implementation details
How should Cluster GOOBS avoid test leakage in its random-to-cluster ratio and live-pool updates?
Real-Time Hard Negative Sampling via LLM-based Clustering for Large-Scale Two-Tower Retrieval
Show answer and example
Answer: Build clusters and pools from training-available content and interactions; fix the ratio by design or validation, not repeated test hits. The limited local MovieLens transition proxy uses genres, not the original multimodal clusters. Full-catalog evaluation is separate from the pool.
Illustrative example: A 1:15 mixture does not mean testing only 16 items, and test targets must not enter pool updates.
Sections in the Chinese source: 核心公式, 本地复现. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Method comparison
What does Cluster GOOBS add beyond hard negatives from the current batch?
Real-Time Hard Negative Sampling via LLM-based Clustering for Large-Scale Two-Tower Retrieval
Show answer and example
Answer: A continuously updated out-of-batch pool is larger and less subject to current batch size and chance; semantic segments keep sampling manageable. Balance stale vectors and false negatives with random draws. Constant sampling cost is not constant total training complexity.
Illustrative example: A batch may lack comparable movies; a cross-batch pool has same-genre options, some of which can be false negatives the user would like.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Basics
Why change Cluster GOOBS's training negative pool rather than the online model?
Real-Time Hard Negative Sampling via LLM-based Clustering for Large-Scale Two-Tower Retrieval
Show answer and example
Answer: The two-tower similarity is unchanged. The training denominator sees harder competitors from a live semantic cluster pool; serving requires no new scorer. The changed learning distribution should be assessed on the full catalog, not just a similarly sampled subset.
Illustrative example: Train a camera positive against similar camera or lens negatives while serving the same two-tower vector index.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why does GenPage post-train full-page generation on long-term reward rather than simply imitate past pages?
GenPage: Towards End-to-End Generative Homepage Construction at Netflix
Show answer and example
Answer: Cloning or SFT reproduces logged display behavior, not necessarily long-term effects of positions and combinations. Page-level reward evaluates the full action while post-training respects business feasibility and cold start. Public proxy lists are not production long-term retention evidence.
Illustrative example: Imitation copies rows of popular items. Adding novel content requires real evidence of sustained exploration, not just more diverse titles.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Why is GenPage's MovieLens core not Netflix's production full-page implementation?
GenPage: Towards End-to-End Generative Homepage Construction at Netflix
Show answer and example
Answer: Public data lack real layout, multiple content formats, and long-term rewards. The fixed candidate protocol only checks actual implemented proxy states, constraints, and updates; a ranked list is not a homepage. Preserve negative local results independently of paper gains.
Illustrative example: A ten-movie list has no carousel, copy, or multi-day reactions and cannot verify the production UI or retention policy.
Sections in the Chinese source: 本地复现. The example illustrates the mechanism.
Method comparison
What evidence do GenPage WBC/SFT and long-term RL use?
GenPage: Towards End-to-End Generative Homepage Construction at Netflix
Show answer and example
Answer: Weighted demonstrations build a usable policy from existing pages. RL adjusts beyond imitation using reliable long-term environments and rewards. Movie proxies do not recover retention, and business and cold-start constraints need separate enforcement.
Illustrative example: Learn a sensible layout quickly from logs; explore more only when long-term feedback supports it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why is a full-page GenPage action closer to the homepage objective than per-item prediction?
GenPage: Towards End-to-End Generative Homepage Construction at Netflix
Show answer and example
Answer: Page utility depends on content types, layout, and interactions. Individually relevant items can form repetitive pages. Condition generation on the selected prefix and evaluate overall reward, accepting harder search and credit assignment without assuming global optimality.
Illustrative example: A page of one movie franchise can be individually relevant but provide no exploration entry points.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Core mechanism
Why need both text-group isolation and smooth spatial bias for multi-instance editing?
Editing Everything Everywhere All at Once
Show answer and example
Answer: Text isolation stops one instruction directly modifying another group's semantics. Smooth spatial bias binds edits to instances while blending at boundaries. Hard cuts can show seams; global text can leak color. This attention control does not train a new editor.
Illustrative example: Keep “left shoe red” and “right shoe blue” semantically separate while blending each localized edit naturally.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Implementation details
Why use at least float32 for ε=10^-30 and avoid arbitrary context-to-vision masks?
Editing Everything Everywhere All at Once
Show answer and example
Answer: FP16 can underflow ε and turn finite log bias into −∞. Softmax and accumulation need at least FP32. The local implementation follows text Case 5: context sees the full visual domain without spatial bias, rather than mixing incompatible diagram or processor defaults.
Illustrative example: Computing log(ε) after underflow changes intended soft decay into a hard exclusion.
Sections in the Chinese source: 核心公式, 本地复现, 原文与代码版本边界. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Method comparison
How does smooth spatial bias differ from a hard mask outside the region?
Editing Everything Everywhere All at Once
Show answer and example
Answer: Log-smoothed masks give finite negative bias that fades near boundaries; hard masks prohibit interactions. Unrelated text groups still use negative infinity, and pixels belonging to other instances are zeroed. Local behavior follows paper_v1, not every author default.
Illustrative example: Weak interactions near a shadow can aid blending while isolating another instance's exclusive pixels.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Visual world models: state, generation, and action conditioning
Basics
How does attribute leakage violate a multi-instance edit?
Editing Everything Everywhere All at Once
Show answer and example
Answer: Different objects may need different changes, but unbound color, shape, or texture can spread to other objects. MICE maintains local assignments and contextual blending without new training. Success is applying each requested change only to the right instance, not just making a prettier image.
Illustrative example: Left cup red and right cup blue should not yield two purple cups.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why does POEM build a partial order from multiple objectives rather than order training sequences chronologically?
POEM: Partial-Order Enhanced Real-Time Sequential Modeling for Recommendation
Show answer and example
Answer: Event time does not measure present-request preference. POEM aggregates objective-relative candidate ranks into a partial order for supervision. Disagreement need not become a unique rank. Existing objective judgments are not independent unbiased user preferences.
Illustrative example: Illustration: An item viewed later can be preferred by click and purchase objectives. Use that relation instead of mistaking browse chronology for preference.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why check POEM pairwise majorities for ties or cycles rather than assume a total order?
POEM: Partial-Order Enhanced Real-Time Sequential Modeling for Recommendation
Show answer and example
Answer: A can beat B, B beat C, and C beat A, and ties have no unique direction. If a sequence is needed, apply a defined stable rule and record it. Never use test labels to choose a favorable ordering.
Illustrative example: Illustration: Breaking a cycle by item ID is an additional rule, not evidence that candidates had a consistent true order.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
What can POEM majority comparison lose versus retaining all conflicting evidence?
POEM: Partial-Order Enhanced Real-Time Sequential Modeling for Recommendation
Show answer and example
Answer: A majority compresses goals to one direction and can lose strongly opposed minority objectives and margins. A partial-order focus differs from representing both sides. The mechanism comparison does not prove universal performance.
Illustrative example: Illustration: Three weak votes for A beat two strong votes for B, hiding the purchase margin.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
How do POEM ranks differ from raw CTR or GMV values?
POEM: Partial-Order Enhanced Real-Time Sequential Modeling for Recommendation
Show answer and example
Answer: Ranks preserve which item comes first within an objective but drop units and magnitude, allowing cross-objective comparisons. They cannot replace all value information, such as one versus a hundred dollars.
Illustrative example: Illustration: A can have slightly higher CTR but much lower GMV. The two rank directions omit 'slightly' and 'much.'
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Core mechanism
Why query multiple teachers on student traces rather than concatenate expert traces?
Show answer and example
Answer: Actual student prefixes reveal visited states. Matched teachers give dense supervision there, reducing trajectory mismatch. Supervision fusion isn't parameter averaging or guaranteed skill retention; local evidence is candidate-level.
Illustrative example: Both teachers can guide a mixed math/code intermediate state absent from their complete answers.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Implementation details
What must MOPD teacher weights and dimensions satisfy?
Show answer and example
Answer: Align vocabulary/candidates and normalize weights by the defined rule. Freeze teachers and update student only. Deterministic mini-suite teachers aren't trained specialists.
Illustrative example: Different vocabulary IDs don't align merely by renormalizing the sum.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Method comparison
Where do probability mixture and parameter averaging merge teachers?
Show answer and example
Answer: Probability mixtures form supervision at shared student states and require teacher inference. Parameter averaging directly combines comparable weights and can disrupt coordination. No superiority without matched experiments.
Illustrative example: Adding output distributions differs from averaging weight matrices.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Basics
Why train independent domain teachers first in MOPD?
Show answer and example
Answer: Joint RL can couple domain gradients. Separate specialists optimize their domains before sharing dense student-prefix supervision. Integration still shares parameters and can interfere.
Illustrative example: Train math and code teachers separately, then guide one student.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Core mechanism
Why generate next intent before retrieval and verification in NEXT instead of linking the most semantically similar videos?
NEXT: Reasoning-Driven Video Recommendation via a Vision-Language Model
Show answer and example
Answer: Similarity describes content, not the next need. A VLM infers what follows, retrieval grounds it, and verification forms a directed edge. Expensive inference runs offline; positive engagement triggers confident edges online. The graph expresses possible continuation, not symmetric similarity.
Illustrative example: Illustration: After a coffee-machine unboxing, extraction instructions may be a better next step than another unboxing.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Implementation details
Why inject NEXT NKG candidates additively only after positive engagement?
NEXT: Reasoning-Driven Video Recommendation via a Vision-Language Model
Show answer and example
Answer: Positive feedback triggers continuation, while additive candidates supplement production retrieval instead of replacing it with uncertain graph edges. The trigger must already be observed; deduplicate and enforce eligibility. A large single-seed proxy change applies to its fixed protocol only.
Illustrative example: Illustration: Add tutorial candidates after a user likes an unboxing, not after a skip; retain other retrieval options.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Method comparison
How does NEXT's intent–retrieve–verify chain improve reliability over asking a VLM to name the next video?
NEXT: Reasoning-Driven Video Recommendation via a Vision-Language Model
Show answer and example
Answer: Intent states the need, retrieval grounds it in the catalog, and verification checks fit, reducing invented titles. The stages can still inherit wrong intent or missed retrieval. A local proxy graph increment does not reproduce full NEXT-8B training.
Illustrative example: Illustration: Infer 'learn how to use it,' retrieve and verify a real tutorial; a bad initial intent can still steer the entire chain wrong.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Basics
Why can't NEXT automatically reverse every directed edge?
NEXT: Reasoning-Driven Video Recommendation via a Vision-Language Model
Show answer and example
Answer: A need after A need not reverse after B. Explicit sequels and inferred needs both create directional relations; serving triggers confident edges after positive engagement. Inferred transitions are not observed consumption.
Illustrative example: Illustration: An unboxing can lead to an extraction tutorial without the tutorial leading back to an unboxing.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why does CMSL's contextual lens not prebucket events into click and purchase types?
CMSL: Constructive Multi-Sequence Learning for Recommendation Systems
Show answer and example
Answer: Fixed event types need not match current latent intent. CMSL learns context-conditioned soft assignment into semantic strands, encodes them separately, and aggregates them using nonsequence context. Check duplication and loss of effective history rather than assuming strands are complementary.
Illustrative example: A kitchen-item click can indicate restocking or preparing to move. Different lenses can weight it differently without a fixed click bucket.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why check width costs separately in CMSL's second-order feature-map approximation?
CMSL: Constructive Multi-Sequence Learning for Recommendation Systems
Show answer and example
Answer: Decomposing the SiLU dot product can reorder multiplication to reduce quadratic dependence on history length. Second-order cross terms grow with feature width, so linear length scaling does not settle FLOPs or constants. The local run checks learned lenses and an HSTU-style path; verify the production kernel separately.
Illustrative example: Doubling history from 1,000 to 2,000 can roughly double the length term, while doubling feature width can expand the feature map by much more.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How do CMSL strands differ from simply adding attention heads?
CMSL: Constructive Multi-Sequence Learning for Recommendation Systems
Show answer and example
Answer: Heads project the same input sequence into different subspaces. A lens first reweights or groups events under context, creating distinct latent sequences before encoding. CMSL's linear HSTU uses a specific kernel approximation; strands do not automatically linearize standard HSTU.
Illustrative example: More heads read the same ledger in different ways. Lenses first reorganize it into separate ledgers, which still need effective encoding.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why does CMSL condition contextual lenses on current nonsequence context?
CMSL: Constructive Multi-Sequence Learning for Recommendation Systems
Show answer and example
Answer: Context determines which historical events explain a latent intent together. Soft routing creates multiple strands for encoding and context-query aggregation, avoiding fixed business categories that hide more specific needs. Human-interpretable themes are not guaranteed; check redundancy and collapse.
Illustrative example: Travel can emphasize light equipment, while a home context emphasizes storage within the same shopping history.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Core mechanism
Why can a guardrail violation veto release after significant improvement in the primary metric?
AgentX: Towards Agent-Driven Self-Iteration of Industrial Recommender Systems
Show answer and example
Answer: Recommendation systems must jointly satisfy target gains and key constraints. A single gain can come at the expense of stability or other important experience. The Evaluation Agent uses online evidence and guardrails to veto release, records both successes and failures, and learns from full traces rather than selecting winners and creating optimistic bias.
Illustrative example: A higher click-through rate does not justify release if critical latency violates its limit. Record the failure cause for the next proposal.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Implementation details
Do policy-update counts from 120 local episodes demonstrate large-model SGPO training?
AgentX: Towards Agent-Driven Self-Iteration of Industrial Recommender Systems
Show answer and example
Answer: No. They verify four-stage execution, asset management, and reuse without Kuaishou's repositories, user traffic, or SGPO training. Average proxy cost is not human efficiency, revenue, or user time. Production gains are author-reported; deterministic perfect scores do not replicate industrial recommendation iteration.
Illustrative example: 108 experiment-knowledge reuses are a local workflow count. Without actual A/B tests, they cannot be converted to annual revenue or production value.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Method comparison
What conclusions do offline implementation checks and production A/B tests support in AgentX?
AgentX: Towards Agent-Driven Self-Iteration of Industrial Recommender Systems
Show answer and example
Answer: Offline checks find code, data, and runtime-contract problems before safe traffic exposure. Online A/B tests measure the primary metric and guardrails in a business environment. The tasks and risks differ, so offline success cannot replace release evaluation. Local small episodes or update counts are not the authors' production evidence.
Illustrative example: Unit tests can establish that recommendation code is ready for further experimentation. User behavior and latency effects require a controlled comparison on corresponding traffic.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Basics
Why does the four-stage loop put failure traces in the experiment knowledge base too?
AgentX: Towards Agent-Driven Self-Iteration of Industrial Recommender Systems
Show answer and example
Answer: Failures reveal implementation limits, guardrail risks, and unsupported hypotheses. Recording only successes creates survivorship bias and encourages repeated mistakes. Brainstorm, Developing, Evaluation, and SGPO connect evidence, code, online checks, and knowledge updates. Distinguish the authors' production system from local control flow.
Illustrative example: If a feature improves offline scores but hurts online latency, recording the failure and conditions can prevent repeating the high-risk proposal.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why distinguish candidate-independent, candidate-dependent, and task tokens before unified interactions?
UniFormer: Efficient and Unified Model-Centric Scaling for Industrial Recommendation
Show answer and example
Answer: They encode common user state, current match, and prediction objective respectively. Explicit roles ease scaling, but joint interactions can make states candidate-dependent; reuse needs masks and architecture.
Illustrative example: A shared age token is reusable, but a state containing one candidate's price is not.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
How avoid token misalignment locally?
UniFormer: Efficient and Unified Model-Centric Scaling for Industrial Recommendation
Show answer and example
Answer: Match role, position, and valid mask to fields, and have each task head read its own state. MovieLens lacks industrial fields and serving.
Illustrative example: Task A reading task B's token is wrong despite matching shapes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
What is the trade-off versus separate models per task?
UniFormer: Efficient and Unified Model-Centric Scaling for Industrial Recommendation
Show answer and example
Answer: Shared interactions can transfer signal and ease maintenance; task tokens condition predictions. Parameter sharing can still cause interference.
Illustrative example: Viewing features may help clicks but conflict with commerce goals.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why tokenize feature roles and tasks in UniFormer?
UniFormer: Efficient and Unified Model-Centric Scaling for Industrial Recommendation
Show answer and example
Answer: One interaction interface carries candidate-independent, candidate-dependent, and task-specific states. Tokenization preserves role distinctions rather than flattening all fields.
Illustrative example: User history, current product, and click objective each get appropriate tokens.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Core mechanism
Why does NOVA separate structural semantics, execution, offline effectiveness, and online impact?
Show answer and example
Answer: Runnable code can still violate intended architecture or harm quality. NOVA uses change history and diagnosis to propose updates, then increasingly expensive checks to reject obvious failures early. Offline success still requires independent online evidence; local execution is diagnostic only.
Illustrative example: Illustration: An incorrectly wired task head can pass tensor checks but fail semantics. A structurally correct change can still regress on validation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Implementation details
What outputs should distinguish actual validation from simulated online impact after NOVA selection?
Show answer and example
Answer: Candidates, shape failures, validation scores, and promotion logs can be measured. Without traffic, online stages are simulations or diagnostics, not deployments. Use test only to evaluate the final selection, and record successful skills and prohibited directions in an auditable form.
Illustrative example: Illustration: Rejecting an intentionally bad shape demonstrates an execution gate. Passing simulated impact is not a user experiment.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Method comparison
Why can't NOVA's semantic architecture checks replace ordinary runtime tests or vice versa?
Show answer and example
Answer: Shape-compatible code can violate task wiring or defining information flow. Semantic checks test intent, execution checks runnable behavior, and offline/online stages test value. Lower training loss does not justify skipping semantics. The local candidate workflow has no production A/B.
Illustrative example: Illustration: A purchase head wired to click labels can train and lower loss; semantic checks should catch it before deployment.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Basics
Why isn't NOVA's architecture gradient an ordinary backpropagation gradient?
Show answer and example
Answer: It summarizes architecture changes, diagnostics, and metrics as an agent control signal. Directional feedback is not a derivative of a differentiable parameter objective; it cannot differentiate code end to end or guarantee improvement.
Illustrative example: Illustration: 'Widening exceeded budget; changing the gate hurt validation' guides a proposal without defining derivatives for Python characters.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why use disentangled SIDs to connect recommendation and generation in RaG?
Show answer and example
Answer: Recommendation picks suitable content and generation controls creative attributes; entangling both can interfere. D-SIDs separate interfaces and cross-domain rewards coordinate what to recommend with how to produce it. The local discrete interface does not demonstrate video quality or full business lift.
Illustrative example: Illustration: A camping theme code supports selection while creative codes control shots or presentation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
What evidence would demonstrate full RaG video generation beyond the local D-SID interface?
Show answer and example
Answer: Provide the real model and revision, conditions, generated videos, runnable training or inference commands, and quality evaluation. Discrete vectors and ranking metrics demonstrate transfer of code components, not execution. Paper online gains do not fill the local gap.
Illustrative example: Illustration: Returning two content/creative integers is a callable interface, not a video; a placeholder filename is not a visual artifact.
Sections in the Chinese source: 本地复现. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
What does RaG cross-domain reward evaluate beyond visual generation quality?
Show answer and example
Answer: It checks recommendation fit as well as visual quality and coordinates selection with creation. Rewards may conflict or lack verification. One proxy score cannot show both personalized relevance and actual visual gains. Locally only the D-SID interface ran.
Illustrative example: Illustration: Beautiful climbing footage can be irrelevant to easy camping, while matching a keyword does not prove watchable realistic video.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Can RaG retrieval hits alone measure new video quality?
Show answer and example
Answer: No. Retrieval evaluates existing candidates, while generated video requires visual, factual, and interest consistency. D-SID bridges interfaces, not success criteria. Local ranking cannot substitute for an unimplemented generator.
Illustrative example: Illustration: Correctly choosing camping can still yield physically impossible footage; pretty footage can also be irrelevant.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why output both a dense vector and future-interest SID tokens?
Show answer and example
Answer: Dense output fits existing rankers but can flatten multiple interests. Item-space SID prefixes provide several actionable clusters without prose. Future-window targets are training only.
Illustrative example: Represent baking and photography with separate prefixes plus a general user vector.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
Does local TokenMinds run full production multi-context decoding?
Show answer and example
Answer: No. A small GRU and SID heads preserve dual output, not Gemini scale, 15 targets, beam 40, or KV refresh serving.
Illustrative example: Dense and code outputs do not establish shared prefill or million-read caching.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
How does a 24-hour multi-target horizon differ from next-view prediction?
Show answer and example
Answer: Several future targets dilute accidental adjacent events, and coarse prefixes represent broader meaning. Future interactions are labels only, not encoder history.
Illustrative example: One misclick should not eclipse the day's other interests.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why output both dense user vectors and SID tokens?
Show answer and example
Answer: Dense features remain compatible with ranking; discrete semantic tokens express multiple future interests. Neither free prose nor one user SID replaces both.
Illustrative example: Retain an overall vector and several prefixes for distinct interests.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Core mechanism
Why not replace cross-lingual semantic aggregation with string voting, and why count aggregation cost?
Cross-Lingual Exploration for Parametric Knowledge
Show answer and example
Answer: Different languages may phrase equivalent facts differently, and superficially similar errors can differ. The method uses an LM for majority/minority-aware aggregation and separate correctness/consistency judgment. All path and aggregation tokens count. This is literature-only locally.
Illustrative example: Two names for one city need entity matching, not separate votes or costs counted from only one path.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Implementation details
Why count more than generated path tokens?
Cross-Lingual Exploration for Parametric Knowledge
Show answer and example
Answer: Include each path's inputs and aggregation model tokens; semantic correctness and consistency also use model calls. Local work did not run the APIs or provide heuristics as substitutes.
Illustrative example: Ten short generations can still produce a long, expensive aggregation prompt.
Sections in the Chinese source: 核心公式, 本地复现. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Method comparison
Why can't a string counter substitute for semantic model aggregation?
Cross-Lingual Exploration for Parametric Knowledge
Show answer and example
Answer: Equivalent meanings can use different strings and equal strings can differ in context. An LM performs majority/minority-aware aggregation and semantic judgments in the paper.
Illustrative example: “United States” and “USA” can be one answer, despite string counts splitting them.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Basics
Why explore multiple languages for one knowledge question?
Cross-Lingual Exploration for Parametric Knowledge
Show answer and example
Answer: Knowledge may be more accessible under some language contexts. Explore across languages then aggregate semantic answers, not mere string translations or presumed-correct majority.
Illustrative example: Distinct expressions may refer to one entity.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Core mechanism
Why inspect both pass@1 and larger pass@k for capability boundaries?
Show answer and example
Answer: First-hit gain may just popularize known paths. Multiple samples probe coverage, then teacher and RL target boundaries. Finite budgets still miss rare paths; local small sampling does not reproduce high-k expansion.
Illustrative example: Measure whether once-never-solved prompts become reachable with broader sampling, counting teacher cost.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Implementation details
Does local pass@8 establish paper pass@256 gains?
Show answer and example
Answer: No. Teacher/student scales and sampling budgets differ. Genuine generation, triggers and cost curves support scaled mechanisms only.
Illustrative example: Failure in eight draws doesn't imply failure after over 200 draws.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Method comparison
How does a boundary curriculum differ from ranking fixed textual difficulty?
Show answer and example
Answer: Reestimate from current policy samples; textual complexity doesn't track mastery. Finite k can miss rare successes, so boundary depends on budget and isn't impossibility.
Illustrative example: A long problem may leave the boundary after learning a new technique.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Basics
Why use pass@k to probe CoBA-RL boundaries before teacher help?
Show answer and example
Answer: Pass@1 can increase by concentrating old successes. Larger-budget coverage finds currently difficult reachable edges, where teacher guidance supplies new routes for RL consolidation.
Illustrative example: Occasional successes becoming frequent differs from learning a previously unseen solution.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why allow each G2Rec item to belong softly to multiple interest prototypes?
Structuring and Tokenizing Distributed User Interest Context for Generative Recommendation
Show answer and example
Answer: The same item can serve several audiences, and hard assignment loses cross-interest relations. Learn soft memberships from the global co-engagement graph and alternate prototype-combined interest tokens with items. Build the graph only from visible training behavior to prevent leakage.
Illustrative example: A camping cooking video can contribute to outdoor and cooking prototypes without forcing either audience into one label.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
What should be checked in G2Rec's prototype means and alternating sequences?
Structuring and Tokenizing Distributed User Interest Context for Generative Recommendation
Show answer and example
Answer: Weight item vectors by membership and divide by total weight with explicit handling for empty or tiny clusters. Align item/profile positions and causal masks; auxiliary targets cannot read future items. Local evaluation uses one positive and 99 negatives, not full-catalog ranking.
Illustrative example: Item A must be followed by A's profile, not B's, even if a misaligned sequence still reduces loss.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
How does G2Rec soft modularity form interest groups differently from a fixed taxonomy?
Structuring and Tokenizing Distributed User Interest Context for Generative Recommendation
Show answer and example
Answer: Taxonomy imposes buckets. Soft modularity learns overlapping communities from co-engagement while accounting for degree background. Communities need not match human categories and can reflect exposure. Graph quality does not guarantee sequence ranking improvement.
Illustrative example: A camera and travel bag can share a community while one book partially belongs to a second one.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why insert interest-profile tokens between G2Rec item tokens?
Structuring and Tokenizing Distributed User Interest Context for Generative Recommendation
Show answer and example
Answer: Item tokens preserve identity; profiles expose shared graph-based interests from multiple memberships. Alternating tokens lets generation use both. Profiles are model structure, not extra consumed events.
Illustrative example: A travel photography book can carry both interests through an adjacent mixed profile instead of its ID alone.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why does JourneyFormer encode event types and time in the guest journey rather than just listing viewed properties?
JourneyFormer: Encoding Airbnb Guest Journey with Sequence Modeling
Show answer and example
Answer: The same property means different things during search, viewing, saving, and booking. Time gaps distinguish one trip from older plans. A shared journey sequence learns relations between stages while retaining both long-term preferences and immediate travel needs. Caching and sparsity reduce cost but do not relax which events were visible at request time.
Illustrative example: Illustration: A user saved a seaside listing last year but now searches downtown and views business apartments. Listing IDs alone do not distinguish the trips; event type and timing can reveal the current one.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why must listing, event type, and time use one version in JourneyFormer's incremental cache?
JourneyFormer: Encoding Airbnb Guest Journey with Sequence Modeling
Show answer and example
Answer: All three determine state, so changing an event type or time encoding makes the old cache inconsistent with new inputs. A causal sequence can contain only events already observed, not future bookings. The local proxy lacks real lodging inventory and full production refreshes, so its latency does not establish online performance.
Illustrative example: Illustration: If a view is corrected to a booking, a stale cache will still treat the user as exploring and may keep recommending. Refreshing the listing vector alone will not fix the event state.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
What does a unified long and short guest journey retain, and what does it cost compared with a recent window?
JourneyFormer: Encoding Airbnb Guest Journey with Sequence Modeling
Show answer and example
Answer: Long history preserves stable destinations and accommodation preferences, while the short window shows the current trip stage. Joint encoding can connect them but adds attention and update costs, motivating caches and sparsity. Older trips can be stale, and more history need not be more accurate. The source's local negative result applies only to its proxy.
Illustrative example: Illustration: A family-room preference may remain relevant during a business trip, but the new context also matters. The newest events alone can forget the stable need, while equal weighting of every old trip can distract.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why encode a guest journey rather than a set of viewed listings in JourneyFormer?
JourneyFormer: Encoding Airbnb Guest Journey with Sequence Modeling
Show answer and example
Answer: Searches, views, saves, and bookings mark different stages, and elapsed time affects whether a need still applies. Type and time embeddings let the same property mean different things in different paths and jointly represent long- and short-term behavior. Event labels are not causal explanations, and a completed past trip need not match current demand.
Illustrative example: Illustration: A seaside listing booked three months ago and an urban apartment saved today should not count equally as unmet needs. A set of listing IDs loses that distinction.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
Why combine surprisal with target entropy rather than always reward surprising tokens?
Show answer and example
Answer: Surprising positions can influence entropy strongly; amplifying them always can overdisperse policy. Select batch quantiles, then adjust advantages based on entropy error. Surprise is not a correct-action label.
Illustrative example: A rare token can need opposite adjustments under an overconfident vs overrandom policy.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
Why identify STARE positions by batch quantiles?
Show answer and example
Answer: Quantiles measure relative salience in the current batch, not an absolute threshold. Fix valid-token masks and probability source; exclude padding. Local checks cover weighting and feedback only.
Illustrative example: Artificial low padding probabilities can shift the critical quantile.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
How does STARE's entropy feedback differ from a fixed entropy bonus?
Show answer and example
Answer: Fixed bonuses always push one direction; feedback reacts to the current–target gap. Targets and measurement matter. Higher entropy is not inherently better and surprise doesn't imply correctness.
Illustrative example: Above-target entropy requires a different control direction from below-target entropy.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
Why focus STARE on surprising tokens rather than equally reinforce a trajectory?
Show answer and example
Answer: The same advantage can affect entropy differently by position. Select critical surprisal quantiles and adjust weights with target-entropy feedback, not terminal reward alone.
Illustrative example: An uncertain branch and punctuation can share reward without serving equal exploration roles.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Core mechanism
Why correct popular edges before offline multi-hop PPR in RankGraph-2 rather than deploy a deeper online GNN?
RankGraph-2: Lifecycle Co-Design for Billion-Node Graph Learning in Recommendation
Show answer and example
Answer: Multi-hop propagation finds indirect links but popular nodes absorb mass and online depth costs latency. Correct transitions and compute PPR offline, then compress serving via residuals and cluster indexing. Popularity correction does not eliminate all exposure bias, and the graph needs refreshes.
Illustrative example: Illustration: A blockbuster can connect unrelated films. Reducing its incoming transition weight lets niche relations survive across hops.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Implementation details
Why renormalize transitions after RankGraph-2 popularity correction before PPR?
RankGraph-2: Lifecycle Co-Design for Billion-Node Graph Learning in Recommendation
Show answer and example
Answer: Reweighting changes row sums. Renormalize to legal transitions or probability mass is spuriously lost or amplified. Check isolates and restart, and validate residual/cluster indexing separately. Designed debiasing does not prove all exposure bias is gone.
Illustrative example: Illustration: A row sum falling to 0.4 would lose 60% of mass each hop without renormalization.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Method comparison
How do offline PPR plus a compressed index and online deep GNN trade cost for freshness?
RankGraph-2: Lifecycle Co-Design for Billion-Node Graph Learning in Recommendation
Show answer and example
Answer: Offline computation amortizes hops and makes lookups light but may lag new edges or current intent. Online propagation is fresher but repeats work per request. PPR iteration convergence is not end-to-end throughput; the proxy is not a full production ablation.
Illustrative example: Illustration: New content with links today waits for the offline refresh, while online propagation can read it at greater serving cost.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Basics
Why combine restart with multi-hop propagation in RankGraph-2 PPR?
RankGraph-2: Lifecycle Co-Design for Billion-Node Graph Learning in Recommendation
Show answer and example
Answer: Restart preserves seed personalization while corrected multi-hop edges expand beyond immediate neighbors. Popularity adjustments reduce mass concentration, but do not prove unbiased relations. Stale graphs and index compression can lose tail paths.
Illustrative example: Illustration: Photography links through lenses to tutorials; restarting avoids drifting entirely to site-wide hits.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why extract query-relevant history and condition item predictions on the entire slate in PIANO?
PIANO: Personalized Reranking via Information Aggregation Node for Music Search Optimization
Show answer and example
Answer: History selection captures current search intent; slate aggregation tells each CTR/CVR prediction which alternatives surround the item. They model different relationships. A change in candidates can change every score, so these are not independent cacheable item values.
Illustrative example: Illustration: A hiking-shoe search reads outdoor history, while the same shoe has different appeal among premium models versus beginner models.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
How avoid leaking gold target position through PIANO list aggregation?
PIANO: Personalized Reranking via Information Aggregation Node for Music Search Optimization
Show answer and example
Answer: The candidate list is allowed, but never order or flag it using real clicks. Labels enter loss only. Mask padding and candidate count, and query only past history. Otherwise IAN can learn a position shortcut.
Illustrative example: Illustration: Always putting the correct product first inflates test results if that layout is repeated; use a serving-equivalent unlabeled list.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
How does PIANO list conditioning change caching and evaluation versus pointwise CTR?
PIANO: Personalized Reranking via Information Aggregation Node for Music Search Optimization
Show answer and example
Answer: Independent scores can be reused for fixed user–item inputs; list-conditioned scores also depend on candidate membership. Extra context may help competition modeling but costs compute and is distribution-sensitive. The public proxy was negative, not universal superiority.
Illustrative example: Illustration: Removing the cheapest item can alter all other predictions. Reusing old scores would ignore the new IAN context.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why does PIANO's IAN read the entire candidate list?
PIANO: Personalized Reranking via Information Aggregation Node for Music Search Optimization
Show answer and example
Answer: Choice depends on substitutes and competitors on the page. Aggregation conditions item CTR/CVR on the current alternatives, while query-related history represents needs. This differs from static item value and changes when the list changes.
Illustrative example: Illustration: A mid-priced shoe can look practical among expensive models but not among cheap ones.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why retain task-private channels and isolate their gradients inside OneRank's unified sequence?
OneRank: Unified Transformer-Native Ranking Architecture for Multi-Task Recommendation
Show answer and example
Answer: Task tokens condition shared features but cannot prevent conflicting losses from competing for shared parameters. Private channels add dedicated capacity that other tasks cannot update directly. Shared inputs still exchange evidence, and conflict is controlled at specific paths rather than assumed absent.
Illustrative example: Illustration: Clicks value novelty and purchases value reliability. Both share user and item history, but click loss must not overwrite purchase-private features.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
How check that OneRank loss Lj cannot update another task's φk without freezing shared θ?
OneRank: Unified Transformer-Native Ranking Architecture for Multi-Task Recommendation
Show answer and example
Answer: Backpropagate one task loss. Other private gradients must be zero; its own private and permitted shared parameters can receive gradients. Detaching the whole backbone loses joint training and detaching only outputs may leave private internal paths. Align parameter groups with tasks.
Illustrative example: Illustration: Click training should preserve purchase-private weights but can update shared fields; zeroing all shared gradients is frozen adaptation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
What differs between OneRank's private channels and MMoE task-specific gates?
OneRank: Unified Transformer-Native Ranking Architecture for Multi-Task Recommendation
Show answer and example
Answer: A gate mixes shared experts that other tasks still update. Private channels own task-specific parameters and block other tasks' gradients. Both retain sharing, not complete independence. The public proxy does not establish that industrial negative transfer is solved.
Illustrative example: Illustration: A purchase gate changes expert weights but clicks can update those experts. Click loss should not change purchase-private parameters.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why put OneRank task tokens inside the encoder instead of using them only to select output heads?
OneRank: Unified Transformer-Native Ranking Architecture for Multi-Task Recommendation
Show answer and example
Answer: Conditioning inside layers changes how each task interprets fields before prediction. Private channels retain specialization, so unified inputs do not mean complete sharing. The presence of a token does not prove it is used.
Illustrative example: Illustration: Purchase can read budget early and clicks can read freshness; a late head gets one already compressed representation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Core mechanism
What does EvoRec's Skill Evolver evolve beyond preserving final model code?
EvoRec: Self-Evolving Agentic Recommender Systems
Show answer and example
Answer: Code and weights do not record how a useful change was found or under which conditions. Skill Evolver distills reusable optimization skills from durable experiment memory for future Research and Code Agents. Skills still require actual execution and validation before generalization claims.
Illustrative example: A sampler helps only the long tail; remember both effective and failed slices, and test on the next dataset rather than copying the champion.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Implementation details
Why retain data protocols and failure conditions in EvoRec memory?
EvoRec: Self-Evolving Agentic Recommender Systems
Show answer and example
Answer: A winning name cannot establish applicability. Bind the candidate to validation, observations, and constraints, including failures, and execute real operators when reusing skills. A registry label is not integration, and three local generations establish only the tested space.
Illustrative example: Record better matched-budget results on sparse histories and that long histories remain unverified, rather than declaring content towers always better.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Method comparison
How does EvoRec skill memory change exploration compared with fixed grid search?
EvoRec: Self-Evolving Agentic Recommender Systems
Show answer and example
Answer: A fixed grid repeats predefined configurations; remembered skills can reprioritize or combine methods from prior feedback. Competition among a few preset skills validates only bounded control flow, not open-ended invention. Select on validation; test must not guide iteration.
Illustrative example: A skill library can prioritize a useful representation transform instead of running ten fixed trials; three skills do not prove arbitrary architecture discovery.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Basics
Why does EvoRec distill optimization skills as well as evolving models?
EvoRec: Self-Evolving Agentic Recommender Systems
Show answer and example
Answer: Code alone loses what was attempted and where it worked, leading to repeated failures. Skill Evolver summarizes durable experiments to inform future Research and Code Agent proposals. A skill is bounded evidence, not a rule exempt from verification.
Illustrative example: Record that strengthening content helps under a validated short-history condition, not just the final two-tower model.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
Why compare both binary KLs rather than only bound a ratio?
Show answer and example
Answer: The same ratio means different movement at different token probabilities. The binary distribution captures token and complement, and KL asymmetry motivates both directions. Passing does not establish token correctness or system-wide lack of bias.
Illustrative example: The same relative shift for a rare and common word is not geometrically equivalent.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
Does KPop retain a token if either KL is small or only if both are?
Show answer and example
Answer: Both directions must be at or below threshold. Stabilize boundary probabilities and track serving vs training source. No local real train/inference engine or MoE cluster.
Illustrative example: Passing only train-to-serve KL still means mask out.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
How does bidirectional binary KL differ from a fixed ratio mask?
Show answer and example
Answer: KL depends on probability level, whereas a ratio mask uses relative multiples. Both directions must pass because KL is asymmetric. A mask doesn't validate synchronization of real asynchronous serving.
Illustrative example: Small KL one way can hide mismatch the other way.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
Why represent one token and the remaining vocabulary as binary in KPop?
Show answer and example
Answer: A sampled ratio ignores absolute mass. Binary comparison includes both token and complement, using forward/reverse KL to gate train–serve mismatch.
Illustrative example: Equal multiplicative changes for rare and common tokens affect complement mass differently.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Core mechanism
Why use identical clips for MVEB's video-only and audio-plus-video comparison?
MVEB: Massive Video Embedding Benchmark
Show answer and example
Answer: Otherwise changes in sample difficulty confound the contribution of audio. Keep clips paired, encoders frozen, and task-specific probes or retrieval metrics fixed; respect each model's frame and audio configuration. An invented audio score for an unsupported model invalidates the comparison.
Illustrative example: Evaluate the same door-knocking video with and without sound, not a different easier clip.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Implementation details
Can a single cosine aggregator implement all MVEB tasks locally?
MVEB: Massive Video Embedding Benchmark
Show answer and example
Answer: No. Official task evaluators include retrieval nDCG@10, clustering V-measure, eight-example-per-class logistic regression, and candidate QA accuracy, each with input and sampling constraints. The local page reviews resources and cost only; no model calls or formal metrics.
Illustrative example: Retrieval must calculate nDCG from relevance labels. Mean cosine is neither a replacement nor QA accuracy.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Method comparison
Is forcing every MVEB video model to take 64 frames fairer than respecting its configuration?
MVEB: Massive Video Embedding Benchmark
Show answer and example
Answer: Not necessarily. MVEB uses 2 fps and up to 64 frames for variable-length models, but follows training configuration for fixed-frame models. Fixed clips are needed for audio/video pairing, and models without audio support must not receive fabricated results.
Illustrative example: Feeding 64 frames into an eight-frame model and calling the failure low embedding quality is not a fair test.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Basics
Why is MVEB's video question answering still an embedding evaluation?
MVEB: Massive Video Embedding Benchmark
Show answer and example
Answer: VCQA embeds candidate answers in a retrieval pool, chooses a match for the video, and reports pool accuracy. It is not free generation. The embedding model is frozen; classification and other tasks use their prescribed probes, so output metrics differ by task.
Illustrative example: Select the closest of four supplied action descriptions rather than freely generating an action and judging its wording.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Core mechanism
What does MSA save and preserve by selecting blocks before exact attention?
Show answer and example
Answer: A cheap block index reduces the historical range; main attention computes exact token interactions inside hits. Summaries select rather than replace tokens. A missed answer block cannot be recovered by exact attention on the rest.
Illustrative example: Choose two of ten pages to read word by word, saving main attention on eight but missing a date if its only page was dropped.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Implementation details
Why doesn't reduced MSA attention pair ratio imply the same speedup?
Show answer and example
Answer: Runtime includes indexing, selection, movement, and kernels. The unfused PyTorch/MPS path has fewer pairs but takes longer. Report pair ratio, PPL, and measured time separately.
Illustrative example: An 80% pair reduction can still slow training if overhead dominates.
Sections in the Chinese source: 本地复现. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Method comparison
Does MSA indexing do the same thing as shrinking main attention KV?
Show answer and example
Answer: The index decides where to read; main attention keeps its own exact Q/K/V inside selected blocks. Compressing KV changes what representations are read, not only where.
Illustrative example: Selecting two original passages preserves different evidence from summarizing the whole text into two passages.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Basics
Is an MSA history block a new layer or a token-axis candidate?
Show answer and example
Answer: Blocks organize history for indexing. A light branch chooses blocks and the main branch reads their tokens. Block selection and exact attention have separate costs.
Illustrative example: Choosing a chapter in a table of contents differs from reading its sentences.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why isn't cached atomic intent equivalent to one user vector?
Atomic Intent Reasoning: Bringing LLM Semantics to Industrial Cross-Domain Recommendations
Show answer and example
Answer: Hierarchical retrievable evidence lets each target select an intent chain. Expensive reasoning happens offline; candidate retrieval/fusion stays online. Local proxy does not verify full LLM intent generation.
Illustrative example: Retrieve camping for a tent and lightweight-equipment needs for a commuting bag from the same memory.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
Why validate Atomic Intent's intent extraction and online aggregation separately?
Atomic Intent Reasoning: Bringing LLM Semantics to Industrial Cross-Domain Recommendations
Show answer and example
Answer: Log-frequency and time decay weight evidence; they do not replace offline LLM extraction of hierarchical intent. Similarity retrieval and multi-head aggregation should consume cached chains. If the local implementation only has deterministic proxy scores, a runnable formula does not establish that it reproduces the LLM extraction.
Illustrative example: Weighting ten views by age produces a number, not an explanation such as traveling light. Check the cache first, then the online selection to locate the two kinds of error.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
How does target-conditioned retrieval in Atomic Intent differ from a static text profile?
Atomic Intent Reasoning: Bringing LLM Semantics to Industrial Cross-Domain Recommendations
Show answer and example
Answer: A static summary compresses needs into one account seen by every candidate. Atomic chains are indexed individually, support incremental updates, and let candidates fetch different evidence. Offline parsing, storage, and retrieval need maintenance; aggregation cannot recover evidence omitted by the index.
Illustrative example: Shoes can retrieve exercise needs while cameras retrieve travel photography needs. A fixed summary that only says the user likes the outdoors loses both specific reasons.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why does Atomic Intent break an action into a hierarchy of intents rather than assigning one broad category?
Atomic Intent Reasoning: Bringing LLM Semantics to Industrial Cross-Domain Recommendations
Show answer and example
Answer: A category records what the user viewed; hierarchical intents also attempt to describe purpose and need, so cross-domain behavior can connect through shared intents. Offline extraction preserves fine-grained cacheable units, and the online target item retrieves relevant chains. An inferred intent can be wrong and is not an explicit user statement.
Illustrative example: Camera browsing could reflect a need for light travel gear or studio image quality. A generic photography label loses the distinction between evidence for a lightweight bag and for studio lighting.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Core mechanism
Why calibrate user preference and tool frequency rather than reward triggers alone?
ToolRec: Calibrated Preference Alignment for Query Recommendation in On-Device Assistants
Show answer and example
Answer: Trigger-only rewards overcall irrelevant tools; user-only optimization may ignore supply/use imbalance. Dynamic weights balance both, but frequency is governance, not quality.
Illustrative example: Live flight questions may need a tool, whereas definitions need not call it to boost frequency.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Implementation details
Does local ToolRec execute real tools?
ToolRec: Calibrated Preference Alignment for Query Recommendation in On-Device Assistants
Show answer and example
Answer: No. MovieLens tests calibration/scoring without private sessions or tool calls. Use versioned training-window frequencies, not future test rates.
Illustrative example: Movie genre counts are not API cost or service load.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Method comparison
How do dynamic frequency weights differ from a fixed positive/negative ratio?
ToolRec: Calibrated Preference Alignment for Query Recommendation in On-Device Assistants
Show answer and example
Answer: Dynamic weights follow tool-use statistics; fixed ratios do not. Frequency is not utility, so do not trade relevance away and report only triggers.
Illustrative example: A low-use tool can get more opportunities only where appropriate.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Basics
Does tool frequency mean more calls are better?
ToolRec: Calibrated Preference Alignment for Query Recommendation in On-Device Assistants
Show answer and example
Answer: No. It is a system calibration signal, not user relevance or successful-call quality. Evaluate use distribution and utility separately.
Illustrative example: Raising an uncommon tool's rate does not make it relevant to unrelated questions.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why retain both host-static and live-dynamic SID rather than overwrite one code?
SSRLive: Live Streaming Recommendation with Dynamic Semantic ID
Show answer and example
Answer: Stable identity and temporary content carry different timescales. Separate codes preserve durable audience fit and current topic for multitask prediction. Dynamic IDs still have extraction and refresh lag.
Illustrative example: Preserve a beauty host's audience profile while indicating today's travel-storage stream.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
What can local MovieLens SSRLive establish?
SSRLive: Live Streaming Recommendation with Dynamic Semantic ID
Show answer and example
Answer: Only core static/dynamic representation and interactions, not live frames, trades, or production multitask logs. Separate task labels and snapshots from future outcomes.
Illustrative example: The next rating cannot be used to construct current dynamic features.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
How does a dynamic SID differ from adding a timestamp?
SSRLive: Live Streaming Recommendation with Dynamic Semantic ID
Show answer and example
Answer: It represents changing content; time alone does not identify that content. A timestamp column is not dynamic semantic generation.
Illustrative example: Knowing it is 8 pm does not reveal the product being demonstrated.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why use static and dynamic semantic IDs for live streams?
SSRLive: Live Streaming Recommendation with Dynamic Semantic ID
Show answer and example
Answer: A host's stable style and current topic change on different timescales. Combine both with interactions for viewing, commerce, and engagement; dynamic updates must be fresh.
Illustrative example: A tech host can currently be traveling outdoors.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Core mechanism
Why distill along actual bidirectional decoding rather than random masks?
Show answer and example
Answer: Confidence decoding visits different contexts and attention conversion can cause forgetting. Score student-visited states with frozen AR targets while retaining its causal condition. Different visible evidence cannot be equated positionwise arbitrarily.
Illustrative example: A confidence-first reveal produces contexts random-mask training might never encounter.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Implementation details
Where should OPDLM compute loss and does teacher update?
Show answer and example
Answer: Use specified masked/supervised positions, freeze AR teacher and align logits to student trajectories. Mini-suite does not fully convert/train a real bidirectional LM.
Illustrative example: Including already determined tokens changes denoising supervision.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Method comparison
Do AR teacher and bidirectional student see the same information at a position?
Show answer and example
Answer: No. AR teacher scores causal history, student denoises visible unmasked tokens. Align positions and state what each sees instead of treating conditioning differences as quality.
Illustrative example: Student may see a revealed token to the right while teacher sees only left context.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Basics
Why distill an AR teacher on a diffusion student's own decoding path in OPDLM?
Show answer and example
Answer: Moving to bidirectional denoising can forget learned behavior. Random masks can miss confidence-decoding states. Actual student rollouts reduce mismatch with inference.
Illustrative example: Train on the reveal order actually visited, not only random masking.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why dynamically balance semantic and collaborative rewards?
Show answer and example
Answer: Different examples need different content/behavior trade-offs. Fixed directions can miss useful nonconvex solutions. Reverse preference reasoning and rejection sampling precede multiobjective alignment, without guaranteeing true preferences.
Illustrative example: A cold item may rely on semantics while a popular one has rich interaction evidence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
How isolate rejection sampling and weight fitting?
Show answer and example
Answer: Use permitted training preferences and validation; never use test feedback to choose reasoning or weights. Local MovieLens does not reproduce the real LLM, private codes, or serving.
Illustrative example: Selecting reasoning after reading a test click leaks the target.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
How do dynamic Pareto weights differ from fixed linear weights?
Show answer and example
Answer: A fixed mixture covers one trade-off and can miss nonconvex frontiers. Dynamic weights adapt, but without comparisons they do not establish global Pareto optimality.
Illustrative example: One user may value semantic novelty, another stable behavioral evidence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why use both semantic and collaborative-ID rewards in TAIJI?
Show answer and example
Answer: Semantics explains content, while IDs reflect platform interactions. They can conflict. Activate recommendation reasoning and balance goals dynamically.
Illustrative example: Similar descriptions need not imply co-preference.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Core mechanism
Why use policy-only statistics and gradually add reference advantage in POLARIS HRI?
POLARIS: Guiding Small Models to Write Long Stories
Show answer and example
Answer: Including reference in mean/SD would change every policy advantage. Use policy-only stats, scale reference gradually from zero, and apply identical rubric. Local component isn't full long-story training.
Illustrative example: Five policy stories determine baseline; reference computes its own advantage against them, not inside their average.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Implementation details
How does POLARIS avoid overflow without changing clipped objective?
POLARIS: Guiding Small Models to Write Long Stories
Show answer and example
Answer: Choose the exact branch by advantage sign in log-ratio space, clear padding/zero advantage first and exponentiate afterward. No extra clamp. Full story training was too costly; tensor gradients aren't capability.
Illustrative example: A capped branch doesn't need to compute a huge exp before taking min.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Method comparison
How does gradual reference injection differ from equal mixing at the start?
POLARIS: Guiding Small Models to Write Long Stories
Show answer and example
Answer: Reference weight rises from zero to avoid early domination; policy stories alone determine mean/variance. Equal mixing changes strength and potentially baselines.
Illustrative example: Reference advantage weight at step zero must be zero.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Basics
Why add a teacher-forced reference story to POLARIS policy stories?
POLARIS: Guiding Small Models to Write Long Stories
Show answer and example
Answer: Long story evaluation is sparse; a reference anchors quality under the same external rubric. It is additional training supervision, not test gold or a supplied copyrighted story.
Illustrative example: Practice with a reference story but generate independently at evaluation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Core mechanism
Why synthesize events and sample targets instead of always taking top-k?
Synthetic Data from Cross-Domain Events for Large-Scale Recommendation Systems
Show answer and example
Answer: Estimate target distributions from overlap and emit ordinary training records for different models. Sampling retains tail opportunities rather than repeating head products, without claiming the records occurred.
Illustrative example: Travel browsing may synthesize adapters or organizers, not only the highest-probability suitcase.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Implementation details
What isolation does conditional SCALR estimation need?
Synthetic Data from Cross-Domain Events for Large-Scale Recommendation Systems
Show answer and example
Answer: Estimate co-occurrence from overlapping train users, not target test events; handle zero denominators and missing coverage. Genre proxies are not Meta conversions or CVR.
Illustrative example: Reading test purchases before building translation probabilities leaks future outcomes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Method comparison
How does data-level transfer differ from adding cross-domain modules to each model?
Synthetic Data from Cross-Domain Events for Large-Scale Recommendation Systems
Show answer and example
Answer: Synthetic events reuse the ordinary target training interface. Model-level transfer is tied to architecture. Shared interfaces ease reuse but do not guarantee equal effects.
Illustrative example: Several rankers can share events rather than each implementing a special attention module.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Basics
Do SCALR translated target events actually occur?
Synthetic Data from Cross-Domain Events for Large-Scale Recommendation Systems
Show answer and example
Answer: No. They are conditional samples for downstream training, not exposures, clicks, or conversions. Keep synthetic identity and weights.
Illustrative example: Generating a target-domain training click on B from a source event on A does not create an audit fact.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Core mechanism
Why don't correct local algorithm answers establish cross-lingual gaps?
XLGoBench: Detecting cross-lingual skill gaps with algorithmic tasks
Show answer and example
Answer: Exact scoring removes judge subjectivity, but validated translations and real model outputs across complexities are needed. Local generation has limited structure, not the full noise/difficulty matrix. Gold stays in the scorer, and five task diagnostics do not measure language accuracy.
Illustrative example: An internal shortest-path solver can answer a graph, without testing whether two natural-language versions are equivalent or equally solvable.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Implementation details
Why does Pr-DSA top-eight scoring ignore order but reject duplicates?
XLGoBench: Detecting cross-lingual skill gaps with algorithmic tasks
Show answer and example
Answer: The target is the complete eight-state set after full probability propagation. Reordering is harmless; repeats hide missing states. Other tasks also use full JSON matching, not partial credit.
Illustrative example: Eight list entries beginning A,A,B may contain fewer than eight distinct states and must be rejected.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Method comparison
Is scorer correctness evidence of multilingual generalization?
XLGoBench: Detecting cross-lingual skill gaps with algorithmic tasks
Show answer and example
Answer: No. It validates evaluation of a supplied prediction. Generalization needs real inputs and outputs across languages, complexity scans, and statistics. Small examples do not substitute for seven languages or human translation checks.
Illustrative example: Solving a 12-node shortest path internally does not show an LM can answer it in a specific language.
Sections in the Chinese source: 本地复现, 复现边界. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Basics
Why use tasks with deterministic algorithmic answers?
XLGoBench: Detecting cross-lingual skill gaps with algorithmic tasks
Show answer and example
Answer: Controlled difficulty and exact answers reduce open-ended judging ambiguity. Compare real models across validated language versions; a scorer alone is not capability evidence.
Illustrative example: A state transition can use the same exact solver after changing its language.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Core mechanism
Why correct Rec-Distill teacher sampling probabilities and distill into an auxiliary tower rather than the serving main tower?
Rec-Distill: An Industrial Distillation Pipeline for Large-Scale Recommendation Models
Show answer and example
Answer: Different sampling makes logits represent different probabilities. Correct to the student convention, then train aux on labels and distillation while transferring through shared layers. Main only uses true labels and serves, reducing direct calibration distortion. Teacher quality and freshness still matter.
Illustrative example: Illustration: A teacher downsampling negatives more aggressively cannot supply raw click probabilities unchanged. Correct it and give soft supervision to aux.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Implementation details
Why report raw student and teacher metrics when the Rec-Distill transferability denominator is small?
Rec-Distill: An Industrial Distillation Pipeline for Large-Scale Recommendation Models
Show answer and example
Answer: η is student distillation gain divided by available teacher gain. Near zero, it is unstable. Teacher is not a serving baseline and arms need matching schedules and candidates. Report negative transfer instead of reframing lower head share as success.
Illustrative example: Illustration: Tiny student changes can yield large ratios when the teacher barely exceeds the student. Show all three raw metrics.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Method comparison
What does main/aux distillation protect versus directly fitting serving main to the teacher?
Rec-Distill: An Industrial Distillation Pipeline for Large-Scale Recommendation Models
Show answer and example
Answer: Main keeps the real-label objective, while aux learns both real and teacher labels and transfers indirectly through the shared backbone. This reduces direct calibration bias but is not full isolation. The strong local teacher still transferred negatively.
Illustrative example: Illustration: Aux can learn relative knowledge while main fits clicks; fitting uncorrected probabilities directly can copy the wrong baseline.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Basics
Why asynchronously share black-box teacher logits across Rec-Distill students?
Rec-Distill: An Industrial Distillation Pipeline for Large-Scale Recommendation Models
Show answer and example
Answer: Running a large teacher for every student call is expensive. Reuse outputs generated during training and let students consume them on separate schedules. Sampling and freshness still need calibration and versioning. Black-box exchange reduces coupling but hides intermediate teacher states.
Illustrative example: Illustration: Two small models can consume one batch of teacher scores without running the teacher themselves, provided sample and version provenance are known.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Core mechanism
Why do formalization workers need separate worktrees and a merge queue before checking the global goal?
Formalizing Mathematics at Scale
Show answer and example
Answer: Isolation reduces concurrent file conflicts, the dependency DAG identifies available declarations, and merge review preserves shared consistency. Local Lean success does not cover the entire textbook. The supervisor checks overall progress and replans. Failed experience can become skills, but a branch is not a merged proof.
Illustrative example: Example: Two workers prove separate lemmas. Check name collisions and main theorem dependencies after integration before claiming full formalization.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Implementation details
Did the local run execute textbook-scale Lean agents or only record the protocol?
Formalizing Mathematics at Scale
Show answer and example
Answer: It records the paper's methods, dependencies, and costs without Lean jobs, paid calls, training, or GPU runs. A DAG state machine cannot reproduce days and huge token budgets per book. Over forty thousand declarations are author results, and not every book finished.
Illustrative example: Example: Scheduling forty goals does not mean forty Lean proofs compiled and passed fidelity review.
Sections in the Chinese source: 本地复现与成本决策. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Method comparison
Why isn't mechanical compilation a substitute for independent fidelity review?
Formalizing Mathematics at Scale
Show answer and example
Answer: The Lean kernel checks proofs of formal statements. Added axioms, sorry, weakened claims, or hidden dependencies can bypass the intended problem. Compilation is necessary but not sufficient; verify the original statement, dependencies, and independent review. Completed scheduling is not mathematical fidelity.
Illustrative example: Example: Replacing all natural numbers with zero can make a theorem easy to prove while failing to formalize the original.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Basics
Why isn't formalizing a whole textbook independent translation of each statement into Lean?
Formalizing Mathematics at Scale
Show answer and example
Answer: Later theorems use earlier definitions and lemmas, and work must preserve shared repository state, compilation, and source meaning. AutoFormBot coordinates workers through these dependencies and merges local results into global goals. Compilation only establishes acceptance by the formal system, not faithfulness.
Illustrative example: Example: A changed foundation can break other proofs even if their branches compiled earlier. Recheck after merging.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why predict auxiliary advertisers and interests in Pinterest Ads LLM rather than replace real-time ad ranking?
Fine-Tuned LLM as a Complementary Predictor Improving Ads System
Show answer and example
Answer: Offline user profiles and cross-source history predict conversion-oriented advertisers and interests for supplementary retrieval and conversion features. SFT predicts the next advertiser; GRPO uses hit position and length rewards. Daily batch inference stays outside requests, while real-time scoring and auction remain.
Illustrative example: Illustration: An overnight camping-brand prediction adds a channel and feature; online conversion scoring and auctions still choose the ad.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
Why check output-length and hit-position rewards together?
Fine-Tuned LLM as a Complementary Predictor Improving Ads System
Show answer and example
Answer: Position rewards put correct advertisers early; length penalties prevent longer lists from gaming hits. Deduplicate parsed outputs and count advertisers and interests separately, with the reduced five-item rule explicit. Preserve validation's SFT or zero-quota choices instead of retuning on test mean rank.
Illustrative example: Illustration: Fifty brands may include the target but violate a five-brand budget; a hit alone would reward that loophole.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
Why separate SFT+GRPO versus SFT from LLM features versus two-tower?
Fine-Tuned LLM as a Complementary Predictor Improving Ads System
Show answer and example
Answer: One compares advertiser generation post-training; the other measures downstream integration. Baselines and metrics differ. GRPO did not improve Recall, feature AUC improved slightly, and the supplemental quota was zero. This is not success at every stage.
Illustrative example: Illustration: A generator can stay flat while its log-probability helps prediction. That does not justify more retrieval candidates or imply Recall improved.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Are Pinterest Ads LLM advertisers and interests one-to-one?
Fine-Tuned LLM as a Complementary Predictor Improving Ads System
Show answer and example
Answer: No. Advertisers are entities and interests are themes; one advertiser spans many topics and one topic many advertisers. Outputs provide different features and retrieval clues. Neither represents an observed conversion.
Illustrative example: Illustration: Camping can involve several brands and a department store can sell camping and home items; the topic is not one brand ID.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why adjust primal–dual decoding multipliers dynamically rather than add a fixed diversity bonus?
Stochastic Primal-Dual Decoding for Multiobjective Generative Recommender Systems
Show answer and example
Answer: Fixed weights cannot react to remaining slack. Per-position scoring combines model relevance with auxiliary contribution and adjusts multipliers so later choices can compensate. No retraining is needed, but candidate availability and local search limit results, with possible relevance loss and no guarantee for arbitrary hard constraints.
Illustrative example: Illustration: If four of five slots include only one new author but two are desired, raise the last-slot incentive; reduce pressure once satisfied.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Why keep mirror-descent multipliers nonnegative and control exponent ranges?
Stochastic Primal-Dual Decoding for Multiobjective Generative Recommender Systems
Show answer and example
Answer: Nonnegative multipliers preserve penalty direction, and positive initialization with multiplicative updates preserves sign. Large steps can overflow or become extreme. Match auxiliary units, R/K targets, and relevance scores; follow actual probability/logit conventions rather than interchanging them.
Illustrative example: Illustration: Raw seconds mixed into probabilities can dominate. Repeated exponential increases can pursue quota at the expense of relevance even without an error.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
Can primal–dual soft objectives replace hard eligibility constraints?
Stochastic Primal-Dual Decoding for Multiobjective Generative Recommender Systems
Show answer and example
Answer: No. Multipliers shift preferences but unavailable or out-of-catalog items must be explicitly excluded. A larger multiplier cannot create feasible items. Soft optimization trades objectives within hard eligibility; no arbitrary-constraint guarantee is reported.
Illustrative example: Illustration: Encourage new authors softly, but mask out-of-stock items rather than let them win after a penalty.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why can primal–dual decoding work without retraining a generator?
Stochastic Primal-Dual Decoding for Multiobjective Generative Recommender Systems
Show answer and example
Answer: Serving search combines original relevance with auxiliary scores and updates multipliers while leaving weights unchanged. It cannot create knowledge missing from the candidates or generator. Measure runtime, relevance loss, and constraint satisfaction separately.
Illustrative example: Illustration: Increase new-author weight when lacking one, without changing the base model's scores.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why does MuChaToR optimize intent, discovery, and dialogue rewards beyond music pretraining and instruction learning?
MuChator: Enabling Active Music Discovery via Conversational Music LLMs in Douyin Music
Show answer and example
Answer: Music pretraining adds domain knowledge and instructions teach response format, but neither resolves immediate preference versus discovery. RL provides feedback for intent, discovery, and dialogue with a reference constraint. The local ranking proxy does not demonstrate actual music conversation quality.
Illustrative example: Illustration: For relaxed commute music without only old songs, reward both mood fit and some discovery rather than merely imitating familiar artists.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
Why can't local MovieLens results validate MuChaToR music pretraining and dialogue RL?
MuChator: Enabling Active Music Discovery via Conversational Music LLMs in Douyin Music
Show answer and example
Answer: The proxy lacks actual music corpora, dialogue turns, and reward environments. Fixed-candidate NDCG measures only the scoring path that ran. Dialogue training requires actual updates and valid logs. Keep negative proxy results instead of replacing them with paper business outcomes.
Illustrative example: Illustration: Genre features can test an interface without proving the model understands harmony or can clarify playlist needs.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
How can MuChaToR discovery reward conflict with intent matching?
MuChator: Enabling Active Music Discovery via Conversational Music LLMs in Douyin Music
Show answer and example
Answer: Strict matching may repeat familiar content, while too much discovery promotes irrelevant novelty. Dialogue quality can favor polished explanations too. Multi-objective optimization and KL balance trade-offs; one reward gain does not establish satisfaction. A movie proxy lacks music dialogue feedback.
Illustrative example: Illustration: Experimental music may be novel but poor for a quiet commute; familiar pop may fit without discovery.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why treat MuChaToR as conversational music recommendation instead of just next-track prediction?
MuChator: Enabling Active Music Discovery via Conversational Music LLMs in Douyin Music
Show answer and example
Answer: Current mood, context, and willingness to explore may differ from history. Dialogue can clarify needs, domain pretraining adds music knowledge, and SFT/RL connect interactions. Fluent discussion is not evidence of taste fit; evaluate separately and ground to a valid catalog.
Illustrative example: Illustration: 'Something relaxing for my commute, but not only old songs' requires both mood matching and exploration.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why does L2Rec use separate user-routed LoRA experts for semantic and behavioral views?
L2Rec: Towards Dual-View Understanding of LLMs for Personalized Recommendation
Show answer and example
Answer: Content and collaborative behavior can complement or contradict each other. Sharing all adapter parameters can let the stronger signal overwhelm the other. The experts learn view-specific low-rank directions, user routing picks mixtures, and a cross-view gate fuses them. Parameter efficiency still needs supervision, and a router that always selects the same expert defeats specialization.
Illustrative example: Illustration: Content can help a new user; rich co-consumption may matter more for an established user. Separate views can adapt before being combined per user.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
Which axes and parameters must remain distinct in L2Rec's BA updates and top-k gating?
L2Rec: Towards Dual-View Understanding of LLMs for Personalized Recommendation
Show answer and example
Answer: Each view and expert owns A and B. Form an update shaped like the original weight, then weight it with the user gate. Select top-k on the expert axis and handle cross-view fusion separately. Shapes cannot reveal semantic/behavioral identity mismatches, and local proxy scores do not establish full expert-path training.
Illustrative example: Illustration: Pairing a semantic expert's A with a behavioral expert's B can pass shape checks but changes the mapping. Selecting users instead of experts implements another algorithm.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
How does user-routed LoRA-MoE change parameter sharing compared with one global LoRA?
L2Rec: Towards Dual-View Understanding of LLMs for Personalized Recommendation
Show answer and example
Answer: A global LoRA gives all users the same update. MoE picks combinations from a shared pool, adding conditional capacity without storing a whole model per user. Sparse training can starve experts, so total parameter count does not establish effective capacity or ranking quality.
Illustrative example: Illustration: Photographers and bakers can use different expert proportions while sharing general experts. Always picking one expert reduces MoE to something close to global LoRA.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why distinguish semantic and behavioral LoRA experts in L2Rec's shared LLM?
L2Rec: Towards Dual-View Understanding of LLMs for Personalized Recommendation
Show answer and example
Answer: Text meaning differs from collaborative consumption. View-specific low-rank updates, user routing, and cross-view gating preserve a common backbone alongside distinct task geometries. Experts are shared, not independent models for each user, and personalization depends on routing and data.
Illustrative example: Illustration: A semantic expert understands suspense; a behavioral expert captures that group's consumption habits. They need not point in the same direction.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Core mechanism
Why does deleting all exposure-related information not necessarily produce a good causal recommendation representation?
Causal Representation Learning for Generalisable Recommendation
Show answer and example
Answer: Some signals relate to both exposure and real outcomes. The information-theoretic objective balances outcome information conditional on confounders against dependence on exposure, aiming to preserve transferable information. Causal interpretation still depends on confounders and identification assumptions; distribution-shift metrics alone do not prove all noncausal information was removed.
Illustrative example: Geography affects both exposure and delivery feasibility. Distinguish targeting preference from real delivery effects instead of deleting every regional signal.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Implementation details
How can an implementation of `I(g(T);Y|Z)−λI(g(T);Z)` avoid using target labels at serving time?
Causal Representation Learning for Generalisable Recommendation
Show answer and example
Answer: Y and Z can enter the training objective and estimator; the deployed representation must be computed solely from T available at prediction. The source reports no extra serving overhead after training, but the local public proxy does not prove that causal components were separated.
Illustrative example: Use clicks to train g(T), then compute it before the current click occurs. Concatenating the observed click at serving would leak the answer.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Method comparison
Where do causal representation constraints act compared with IPS weighting?
Causal Representation Learning for Generalisable Recommendation
Show answer and example
Answer: Representation constraints change what the encoder stores, while IPS changes how much observed samples contribute to loss. The former needs confounder representations and information estimates; the latter needs valid propensities and support. Proxy results do not establish that one is generally better.
Illustrative example: Weighting rare impressions does not stop the model reading popularity. Suppressing popularity does not create missing unexposed samples.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Basics
Why does Causal Representation both retain outcome information and suppress confounding information?
Causal Representation Learning for Generalisable Recommendation
Show answer and example
Answer: Suppressing exposure information alone can remove genuine preference; preserving outcome prediction alone can exploit exposure shortcuts. The dual objective uses conditional outcome information and a confounding penalty to retain generalizable signals. These constraints do not establish causal identification without variable and data assumptions.
Illustrative example: Removing popularity entirely can lose demand information; keeping it unconditionally can copy the old exposure policy.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why serialize query and item together in QGS and still add feature interactions?
From Item-Only to Query-Item: Query-Conditioned Generative Search with QGS in Quark
Show answer and example
Answer: Joint serialization conditions generation on search query and history from the first item path, not just a general user vector. History encoding controls context cost and HFG-Attention adds sparse heterogeneous crosses. Its efficiency is architecture-specific, not proof every HSTU is linear.
Illustrative example: Illustration: 'Travel camera' and 'indoor lighting' steer different paths; device and region interactions can further influence candidates.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Why doesn't local QGS +0.73% NDCG validate full generation and a linear kernel?
From Item-Only to Query-Item: Query-Conditioned Generative Search with QGS in Quark
Show answer and example
Answer: A shared scorer proxy changes rank scores without production queries or full kernel throughput measurements. Verify joint input, legal generation, field interaction, and attention separately. Hit declined and should also be reported.
Illustrative example: Illustration: A cross term on movie-transition scores can change NDCG without producing SID decoding traces.
Sections in the Chinese source: 本地复现. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
Why don't Linear HSTU and ordinary HSTU have the same complexity just because they share a name?
From Item-Only to Query-Item: Query-Conditioned Generative Search with QGS in Quark
Show answer and example
Answer: Linear forms need a specific kernel approximation or reordering. Standard SiLU(QKᵀ)V still has pairwise interactions, and HFG fusion adds separate costs. Do not invent kernel details absent from the source or omit dimensions and fields.
Illustrative example: Illustration: Removing softmax does not eliminate the N×N matrix; the actual rewrite or sparse execution matters.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Why does QGS conditional probability describe a complete path rather than justify greedy per-step choices?
From Item-Only to Query-Item: Query-Conditioned Generative Search with QGS in Quark
Show answer and example
Answer: p(y|q,h) multiplies probabilities conditioned on prior generated tokens. Locally best choices need not maximize joint probability. Accumulate log probabilities under legal catalog paths. This identity alone does not establish the full search path in a local proxy.
Illustrative example: Illustration: A slightly stronger first token may have weak legal continuations; another branch can win overall.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why does DeGRe use offline beam lookahead but only a greedy generator online?
DeGRe: Listwise Generative Reranking with Offline Lookahead Distillation
Show answer and example
Answer: Lookahead scores future list value across prefixes but is costly online. Offline search finds valuable lists and distills value distributions over remaining candidates under each prefix into soft targets. Greedy serving approximates future effects; it learns estimated values, not a guaranteed optimum, and may inherit evaluator bias.
Illustrative example: The highest-scoring first item can force redundant later choices. Offline search can find a better opening and teach it without online permutation search.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Why mask selected items and normalize DeGRe's q_t over candidates remaining under the same prefix?
DeGRe: Listwise Generative Reranking with Offline Lookahead Distillation
Show answer and example
Answer: The target represents legal next choices. Repeated items must be excluded, scores must share a prefix, and target softmax and student masks must agree. Greedy serving still updates selected state step by step without a beam.
Illustrative example: After movie A is selected, step two distributes probability among other movies. Keeping A in the denominator can train repeated recommendations.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
How are DeGRe dense soft labels richer than distilling only the best beam list?
DeGRe: Listwise Generative Reranking with Offline Lookahead Distillation
Show answer and example
Answer: A winning list supplies one target per step. Dense labels distribute probability across all remaining candidates under that prefix by lookahead value, with list weights emphasizing better trajectories. Denser targets can also spread evaluator bias.
Illustrative example: Two good books can both get high target probabilities rather than one winner; a popularity-biased evaluator can still mis-score both.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why use DeGRe's offline cumulative-return evaluator instead of only the next logged item?
DeGRe: Listwise Generative Reranking with Offline Lookahead Distillation
Show answer and example
Answer: A point label describes one logged path, not the future list value of alternatives. Beam search uses prefix-dependent lookahead values and distills them into the generator. This expands beyond single-path imitation but can carry evaluator errors.
Illustrative example: A basic tutorial can enable useful advanced items despite a low immediate click probability.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Core mechanism
Why check metric, stage, unit, and direction even when the source and logs exist?
ScientistOne: Towards Human-Level Autonomous Research via Chain-of-Evidence
Show answer and example
Answer: A log can contain training, ablation, and formal evaluation values that are not interchangeable. Match each claim to evidence with consistent identity and protocol, then check recomputability and method–code consistency. Provenance rate is empty when there are no quantitative claims, not perfect. A source's existence does not show it supports the claim.
Illustrative example: A log may record lower training loss while a report claims higher test accuracy. Traceability to a number is not matching evidence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Implementation details
How should five zero-mean recomputations, no quantitative claims, and tied votes be handled?
ScientistOne: Towards Human-Level Autonomous Research via Chain-of-Evidence
Show answer and example
Answer: For a deterministic zero mean, require exact zero without a loose epsilon. CPR is null without quantitative claims, not perfect. No majority or abstention leaves the result unverified. I1 uses population standard deviation for tolerance. Trusted evaluation and log provenance are ensured upstream, and fixtures do not verify the whole study.
Illustrative example: A draft with no numbers does not earn CPR=1 from a zero denominator. Opposing votes from two reviewers do not default to majority approval.
Sections in the Chinese source: 核心公式, 本地复现, 结果与边界. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Method comparison
Why are numeric CPR and scholarly citation authenticity different issues?
ScientistOne: Towards Human-Level Autonomous Research via Chain-of-Evidence
Show answer and example
Answer: CPR checks whether quantitative claims trace to numerically matching execution logs with consistent metric, stage, unit, and direction. Citation verification separately requires retrieval and semantic disambiguation. A number can match while a citation is fabricated, and a real paper may not support the claim. Neither check replaces the other.
Illustrative example: Training loss cannot support test accuracy even if the numbers happen to match. A real paper with the same title does not show it reported the claimed result.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Basics
Why does Scientist-One make research claims independently traceable and checkable?
ScientistOne: Towards Human-Level Autonomous Research via Chain-of-Evidence
Show answer and example
Answer: Automated research can generate code, charts, and conclusions whose reported numbers lack actual execution or do not match source definitions. Scientist-One connects claims, evidence, and run records so readers can inspect their origin. This chain is an audit entry point, not proof by hash; still check contents, units, and experiment stage.
Illustrative example: An accuracy gain should trace to the corresponding split and run. Finding a hashed file does not show the value inside is accuracy.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why use one cross-page story and task prompts for item, carousel, and search?
TubiFM: Unified Item, Carousel, and Search Ranking for Streaming Discovery
Show answer and example
Answer: Separate page models miss links between events. One sequence shares the journey, while prompts identify scoring targets. Shared input does not equal shared objectives or calibrated candidate types.
Illustrative example: Use suspense search and carousel browsing as common context, then specify whether ranking films, rows, or results.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
How does local task serialization differ from the production foundation model?
TubiFM: Unified Item, Carousel, and Search Ranking for Streaming Discovery
Show answer and example
Answer: Fixed public candidates test model state and task-conditioned scores, not private features, scale, or traffic. Align task tokens, histories, and labels.
Illustrative example: Three prompts must change execution, not just log labels.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
What is the trade-off between one task-prompted model and separate page models?
TubiFM: Unified Item, Carousel, and Search Ranking for Streaming Discovery
Show answer and example
Answer: Sharing can transfer signals and simplify maintenance, but may cause interference. Evaluate each task under matched budgets.
Illustrative example: Long-term home-feed taste should not override a precise search intent automatically.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why serialize cross-page activity as a Tubi user story?
TubiFM: Unified Item, Carousel, and Search Ranking for Streaming Discovery
Show answer and example
Answer: Item, carousel, and search activity relates across the journey. One history and task prompts share evidence while preserving task-specific ranking.
Illustrative example: Search then browse a collection to explain what film the viewer now wants.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
What types of forgetting do Memento's representation and data-replay paths address?
Memento: Personalized RAG-Style Long-Retention Data Scaling for Online Ads Recommendation
Show answer and example
Answer: Truncated inference history loses distant interests; the representation path retrieves relevant nonredundant events and conditions predictions. Single-use training samples can be forgotten through parameter updates; the data path replays old samples related to the current batch. Local history retrieval does not validate both paths.
Illustrative example: Illustration: Retrieve last year's photography click as input; replay old training examples to relearn a rare category. Reading an old vector is not training replay.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why can't Memento's local metrics verify Ember transformations and data replay?
Memento: Personalized RAG-Style Long-Retention Data Scaling for Online Ads Recommendation
Show answer and example
Answer: Only Representation Memento's MMR ran. Ember affine/quadratic transforms, production retrieval, and historical replay were not reproduced. Broader claims need learned conditional parameters, actual replay updates, and matched controls. Choose λ on validation, not test.
Illustrative example: Illustration: Better NDCG after recovering an old photography event supports input evidence, not claims about forgetting after SGD without replay updates.
Sections in the Chinese source: 本地复现. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does query-conditioned MMR in Memento trade relevance and redundancy against LastN?
Memento: Personalized RAG-Style Long-Retention Data Scaling for Online Ads Recommendation
Show answer and example
Answer: LastN uses recency; MMR picks query-relevant events while avoiding duplicates of already selected history. It can recover distant evidence but depends on query quality, with λ balancing relevance and coverage. Local results cover retrieval, not Ember or data replay.
Illustrative example: Illustration: Ten recent lens views are redundant; MMR can keep one and retrieve an older camera-use event, or choose poorly if the query is wrong.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why are Representation Memento and Data Memento different memory stores?
Memento: Personalized RAG-Style Long-Retention Data Scaling for Online Ads Recommendation
Show answer and example
Answer: Representation memory retrieves long-term user evidence for a current prediction. Data memory finds forgotten examples related to the current batch and updates parameters again. Storage, updating, and evaluation differ, so implementing either does not complete both loops.
Illustrative example: Illustration: Reading old photography history today is representation memory. Replaying training examples to prevent forgetting a user segment is data memory.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Core mechanism
Why freeze the strong document tower during HARNESS-LM refinement and train only the small query encoder?
HARNESS-LM: A Three-Phase Training Recipe for Harnessing SLMs in Sponsored Search Retrieval
Show answer and example
Answer: Document vectors can be precomputed; query encoding is latency-sensitive. Align the small tower to the teacher query space, then contrastively refine in the fixed document space to avoid index drift and costly rebuilds. Freezing limits adaptation and does not guarantee teacher quality.
Illustrative example: A student searches an existing million-ad coordinate system instead of changing it and invalidating the index.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Implementation details
Why verify no teacher document gradients during HARNESS-LM refinement?
HARNESS-LM: A Three-Phase Training Recipe for Harnessing SLMs in Sponsored Search Retrieval
Show answer and example
Answer: A frozen document tower preserves the existing index while the student adapts. Changing documents invalidates the no-rebuild contract. Check gradients or parameter fingerprints rather than optimizer names, and do not infer online index deployment from alignment.
Illustrative example: After one step query parameters may change but document parameters must not; moving document vectors breaks an old index.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Method comparison
Why are HARNESS-LM L2 alignment and subsequent InfoNCE refinement not redundant?
HARNESS-LM: A Three-Phase Training Recipe for Harnessing SLMs in Sponsored Search Retrieval
Show answer and example
Answer: L2 initializes query coordinates close to the teacher; InfoNCE directly distinguishes relevant from competing fixed documents rather than minimizing coordinate error. Local loss convergence alongside retrieval regression shows that these are distinct signals.
Illustrative example: A vector closer to the teacher can still misorder similar ads; contrastive refinement explicitly compares them.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Basics
Why can HARNESS-LM use a large offline document encoder and a small online query encoder?
HARNESS-LM: A Three-Phase Training Recipe for Harnessing SLMs in Sponsored Search Retrieval
Show answer and example
Answer: Document work is amortized through reuse, while live queries are encoded on demand. Align the student query space, then refine against frozen documents. A smaller student can still lose complex intent distinctions.
Illustrative example: Precompute a million ad vectors overnight while processing each live query in milliseconds.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why teach tail items from head items asymmetrically and stop teacher gradients?
Show answer and example
Answer: Heads have richer interactions. Weight tail-to-head transfer while preventing tail noise from altering teachers. Cluster/individual views and activity gates preserve distinctions.
Illustrative example: A new shoe borrows common knowledge without a few accidental clicks rewriting the best-selling cluster.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
Where are AKT-Rec's two activity gates?
Show answer and example
Answer: First blend cluster and individual item embeddings; second blend cluster and instance histories based on user/item/cross activity. Do not collapse them or detach both contrastive sides.
Illustrative example: A new item can need cluster embeddings and a sparse user can independently need cluster history.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
Does asymmetric transfer just add more contrastive examples?
Show answer and example
Answer: No. It weights tail-to-head learning and detaches teacher-side gradients, unlike symmetric adjustment. This assumes heads are more reliable and may fail in some clusters.
Illustrative example: A new niche film can learn from established neighbors without pulling them toward its noisy clicks.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why retain cluster and individual views?
Show answer and example
Answer: Clusters share knowledge where feedback is scarce; individual embeddings preserve behavior differences. Activity gates balance them and soft orthogonality encourages complementarity.
Illustrative example: Two sci-fi films can share a cluster while preserving action versus philosophical audiences.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Core mechanism
Why add A/A′ predictability checks to LLM ad retrieval instead of only measuring Recall?
LLM Retrieval for Stable and Predictable Ad Recommendations
Show answer and example
Answer: Small edits preserving creative semantics can still change delivery sharply through IDs and sparse signals, without showing up in Recall. A primary/shadow pair measures conversion and exposure drift; hierarchical semantic attributes and an ad graph offer transferable connections. Control sampling noise before declaring a small difference a failure.
Illustrative example: Illustration: Rephrasing an ad without changing its meaning can nearly erase its impressions while standard hit rates remain good. A paired shadow ad specifically tests such sensitivity.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Implementation details
Why don't three identical tie-breaking seeds prove zero training variance in local LLM ad retrieval?
LLM Retrieval for Stable and Predictable Ad Recommendations
Show answer and example
Answer: The model was not independently retrained for each seed, and there were no score ties to break. Identical results establish deterministic postprocessing only. Report A/A′ attribute and score drift separately from Recall and NDCG.
Illustrative example: Illustration: Changing the tie-breaking seed for one checkpoint on a list with no ties naturally produces the same list, not three independently stable training results.
Sections in the Chinese source: 本地复现. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Method comparison
What does an LLM semantic graph add for new ads and stability compared with a collaborative item graph?
LLM Retrieval for Stable and Predictable Ad Recommendations
Show answer and example
Answer: Collaborative edges come from co-consumption. Semantic attributes connect behavior-sparse creatives and can reduce drift under meaningless rewrites. Hallucinations require confidence and expansion budgets. The local result is a fixed-budget routing increment, not end-to-end ad effectiveness or strong evidence about training variance.
Illustrative example: Illustration: An ad with no co-click edge can connect by brand and use. A misidentified brand can also expand to wrong candidates, so more edges alone are not better.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Basics
Why must an A/A′ shadow-ad perturbation preserve ad meaning?
LLM Retrieval for Stable and Predictable Ad Recommendations
Show answer and example
Answer: It tests stability to irrelevant changes. Meaningful edits can reasonably change distribution. Alter formatting or equivalent wording to probe reliance on IDs or superficial sparse features. Stability is distinct from relevance: identical wrong retrieval is stable too.
Illustrative example: Illustration: Punctuation and equivalent year formats should largely preserve audience; replacing children's shoes with hiking boots genuinely changes the product.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Core mechanism
Why estimate PEARL percentiles via repeated comparisons rather than regress raw behaviors?
Show answer and example
Answer: Raw values differ in scale and bias. Comparing a behavior against references estimates its relative position, with more independent comparisons reducing variance. Value weighting and long-term co-training connect objectives. Percentiles depend on the reference and are not causally debiased value.
Illustrative example: Illustration: Thirty seconds means different things for a 15-second clip and a ten-minute video; compare within suitable references.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Implementation details
Why check dependence among PEARL comparisons before using p(1−p)/N?
Show answer and example
Answer: The expression assumes appropriate Bernoulli comparisons and sampling. Repeated or correlated references do not add N independent observations. Record weights, bootstrap, and co-training provenance; sampling variance is not training variance. Do not use test references to choose targets.
Illustrative example: Illustration: Comparing the same video 100 times does not reduce uncertainty tenfold as 100 independent references might.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Method comparison
How do repeated pairwise comparisons trade scale and variance against raw-label regression?
Show answer and example
Answer: Regression preserves units but is sensitive to extremes. Comparisons organize relative position across scales but lose absolute gaps and introduce sampling error. Under independence, larger N reduces variance; bootstrap does not remove systematic bias. Neither objective is universally superior.
Illustrative example: Illustration: 100 and 101 seconds can differ in percentile despite a small raw gap, while a huge anomaly no longer dominates linearly.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Basics
Why identify PEARL's reference distribution when reporting a percentile?
Show answer and example
Answer: A percentile is the fraction of reference outcomes exceeded, so users, context, and windows change its meaning. More comparisons reduce sampling noise, and value weights connect goals. Relative rank does not remove logging bias or prove causal incrementality.
Illustrative example: Illustration: Thirty seconds can be high for short clips and low for long tutorials; a wrong reference reverses interpretation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why does FLUID keep slice and room LUCIDs and phase out item IDs?
Show answer and example
Answer: Short-lived rooms accumulate little stable ID feedback. Slice codes describe current content and room codes aggregate stable history. Late fusion keeps the time scales distinct; add slices, remove IDs, then add rooms to ease training transitions. Content codes can still collide or go stale.
Illustrative example: A sportswear room may currently sell shoes; room and slice codes describe different evidence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
Why index each LUCID embedding by the full prefix rather than an isolated codeword?
Show answer and example
Answer: A later code depends on earlier residual choices; cumulative prefix indexes retain path identity. Isolated codewords merge different branches. Final candidate tokens must contain only room and slice codes to implement item-ID removal.
Illustrative example: Fine code 3 after beauty and after travel needs separate embedding entries.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
What shared information does FLUID use after removing transient item IDs instead of randomly initializing new room IDs?
Show answer and example
Answer: Random IDs learn only after interactions. Semantic prefixes reuse cross-domain content patterns immediately, useful for short-lived rooms, but lose distinct identity memory. Removing IDs lowered overall local quality; report that alongside improved freshness.
Illustrative example: A ten-minute stream has little time to train an ID. Content can place it immediately but may not distinguish hosting styles.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why treat live slices and room-history summaries as distinct tokens in FLUID?
Show answer and example
Answer: The slice LUCID reflects fast-changing current content while the room LUCID aggregates stable historical semantics. Late fusion prevents historical majority from erasing fresh themes. A room vote can miss minority changes and cannot replace the slice.
Illustrative example: A beauty creator currently discussing travel storage has different room and slice signals.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Core mechanism
Why pre-encode frequent phrases and fall back to hashed Engram?
Memory Grafting: Scaling Language Model Pre-training via Offline Conditional Memory
Show answer and example
Answer: An offline model encodes common n-grams into frozen memory; the recipient tries longest-suffix exact lookup. Trainable hash memory covers misses. Projection and contextual gating decide how to use the result, separating offline knowledge cost from online access.
Illustrative example: A common trigram hits the bank while a rare combination falls back to hashing; either can be gated away.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Implementation details
Which phrase state is stored in the offline bank?
Memory Grafting: Scaling Language Model Pre-training via Offline Conditional Memory
Show answer and example
Answer: Freeze the final token's hidden state at the specified intermediate grafting layer. The recipient uses separate projection, context gate, and short convolution. End-to-end bank retraining changes the method.
Illustrative example: Store token four's state for a four-word phrase rather than arbitrarily averaging all four.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Method comparison
How does grafted memory differ from Engram trained from scratch?
Memory Grafting: Scaling Language Model Pre-training via Offline Conditional Memory
Show answer and example
Answer: A pretrained model encodes and freezes the exact bank; longest-suffix lookup precedes trainable hash fallback. Table size alone is not the difference, and local results do not beat equal-parameter Engram.
Illustrative example: An absent exact phrase uses the explicit hash fallback, not a fabricated value.
Sections in the Chinese source: 背景与主要改动, 本地复现. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Basics
What are the roles of grafting model and recipient?
Memory Grafting: Scaling Language Model Pre-training via Offline Conditional Memory
Show answer and example
Answer: The grafting model supplies offline intermediate phrase states; the recipient reads this frozen memory online without rerunning the teacher for each phrase.
Illustrative example: Encode phrases once and look up values later.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Core mechanism
Why consider Self–Peer disagreement in MDCNS rather than always choosing Self's highest-scoring negative?
Divergence Meets Consensus: A Multi-Source Negative Sampling Framework for Sequential Recommendation
Show answer and example
Answer: A high Self score may be bias or an unlabeled positive; repeatedly choosing it reduces diversity. A heterogeneous Peer adds another view, disagreement finds informative candidates, and Teacher distills relationships across the full pool to Self. Disagreement does not establish a negative label; manage false negatives and training stability.
Illustrative example: Illustration: If Self over-scores popular items and Peer disagrees, the discrepancy reveals uncertainty, not proof that every disputed item is irrelevant.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Implementation details
Why compute MDCNS Teacher KL over the whole pool instead of the chosen negative alone?
Divergence Meets Consensus: A Multi-Source Negative Sampling Framework for Sequential Recommendation
Show answer and example
Answer: One negative supplies a local pair; the full Teacher distribution conveys relative difficulty across candidates. Align Self/Peer columns by identity and preserve temperature and τ² scaling. Build unobserved candidates at the training cutoff rather than using test answers to filter them.
Illustrative example: Illustration: Even if one pair is sampled from 30 candidates, KL can convey which of the other 29 are close. A single scalar normalized to one contains no distributional knowledge.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Method comparison
Why sample randomly from top M in MDCNS instead of DNS's fixed highest-score choice?
Divergence Meets Consensus: A Multi-Source Negative Sampling Framework for Sequential Recommendation
Show answer and example
Answer: Always taking the hardest candidate can repeatedly chase one outlier or false negative. Sampling within top M broadens hard-region coverage. Disagreement reranking and multiple losses reduce dependence on one model but still depend on pool quality. The small Markov backbone supports the sampling result only, not large sequence-model reproduction.
Illustrative example: Illustration: If one popular film always tops the negatives, sampling among five hard items avoids centering every update on that suspected false negative.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Basics
Why does MDCNS use Self, Peer, and Teacher for negatives?
Divergence Meets Consensus: A Multi-Source Negative Sampling Framework for Sequential Recommendation
Show answer and example
Answer: One model's high score can reflect bias or a true unlabeled positive. Peer provides a heterogeneous judgment, Teacher aggregates both, and disagreement identifies hard informative items. False negatives remain possible; distillation over the pool conveys relations a single selected negative misses.
Illustrative example: Illustration: Self and Peer can disagree about a camera's relevance. A missing interaction does not show the user dislikes it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why does GrowthGR use ItemLTV incremental value and multi-objective post-training instead of only increasing new-item exposure?
Towards Sustainable Growth: A Multi-Value-Aware Retrieval Framework for E-Commerce Search
Show answer and example
Answer: Exposure is a means, not an outcome. ItemLTV estimates changes in future orders or value relative to what would happen anyway; multi-objective training balances click, purchase, and coverage. Clip rare high-value weights to limit instability. Counterfactual assumptions and rewards need validation; more new items do not prove growth.
Illustrative example: A first exposure can open a sustained use case or cause one accidental click; equal traffic treats these differently valuable opportunities alike.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Why clip GrowthGR rarity weights based on `−log πold` and fix the reference policy?
Towards Sustainable Growth: A Multi-Value-Aware Retrieval Framework for E-Commerce Search
Show answer and example
Answer: Tiny old probabilities create extreme weights, high variance, and single-sample dominance. πold must match the sample's generating policy and context, not current probabilities. Policy ratios and reward weights have different roles; separate ItemLTV proxies from real orders.
Illustrative example: Clipping can stop one noisy low-probability example from dominating a batch without making its value estimate reliable.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
How do GrowthGR rare high-value weights differ from a uniform novelty reward?
Towards Sustainable Growth: A Multi-Value-Aware Retrieval Framework for E-Commerce Search
Show answer and example
Answer: A novelty reward uses newness alone. Rarity weights use previous-policy probability and multi-objective value to emphasize potentially useful low-opportunity items. Low probability is not quality; clipping limits amplified noise. Neither alone proves growth.
Illustrative example: Two new items with little exposure should not get identical boosts if estimated long-term relevance differs, though a bad estimate can be amplified.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Why use ItemLTV instead of immediate new-item purchases in GrowthGR?
Towards Sustainable Growth: A Multi-Value-Aware Retrieval Framework for E-Commerce Search
Show answer and example
Answer: Low initial exposure leads to few immediate purchases and can hide effects of first clicks on later orders. ItemLTV separates baseline and increment; MoPO aligns generation with multiple behavioral goals. Predicted incremental value is not inherently causal without credible assumptions and feedback.
Illustrative example: A click today might lead to a later series purchase despite no same-day conversion.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Core mechanism
Why does DADF freeze the mature first-stage model and learn a multiplicative factor?
DADF: A Distribution-Aware Debiasing Framework for Watch-Time Regression in Recommender Systems
Show answer and example
Answer: Retraining can damage established capabilities or interfaces. Distribution-conditioned proportional residuals adapt to inputs and base predictions while preserving the scalar output. Tiny predictions can destabilize ratios, so check denominator protection and conditional distribution calibration.
Illustrative example: If two groups each start at 10 seconds but one is overestimated and one underestimated, learn distinct factors instead of adding five seconds everywhere.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Implementation details
Why check small denominators in DADF's `y/max(y0,ε)` even with frozen y0?
DADF: A Distribution-Aware Debiasing Framework for Watch-Time Regression in Recommender Systems
Show answer and example
Answer: Freezing stabilizes supervision but tiny predictions can produce huge noisy ratios. Record epsilon and use the same baseline version for correction inputs, ratio targets, and output multiplication. Do not update the first-stage model while claiming second-stage-only training.
Illustrative example: An observed 10 seconds divided by a 0.01-second baseline creates a huge factor that is not necessarily reliable preference evidence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Method comparison
How do DADF's multiplicative and additive residuals affect scale differently?
DADF: A Distribution-Aware Debiasing Framework for Watch-Time Regression in Recommender Systems
Show answer and example
Answer: Multiplication expresses proportional error and therefore larger absolute adjustments for large predictions; addition expresses fixed-unit error. Choose based on error structure. The local ranking was unchanged, so calibration changes are not automatic recommendation gains.
Illustrative example: A 10% underestimate at 10 and 100 seconds suits multiplication; a consistent five-second error suits addition. A global multiplier preserves ranking.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Basics
Why treat DADF's mature predictor as a baseline and learn a relative correction?
DADF: A Distribution-Aware Debiasing Framework for Watch-Time Regression in Recommender Systems
Show answer and example
Answer: The predictor already captures stable relationships. A separate factor focuses on conditional systematic errors with the same scalar interface and less risk than full retraining. Near-zero or badly misspecified predictions are harder to correct, and factors cannot recreate missing information.
Illustrative example: Correct a 10% long-video underestimate without discarding every learned user-content relationship.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
Why normalize GPRL advantages by dimension before context aggregation?
Show answer and example
Answer: Raw sums let large scales or variance dominate and hide tradeoffs among helpfulness, format and brevity. Standardize, weight by context and monitor drift. Normalization isn't value prioritization; exact accuracy isn't a multi-dimensional evaluation.
Illustrative example: Separate a 100-point metric's units from how much users want it weighted.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
Does local GPRL exact-answer regression refute the paper's open-ended results?
Show answer and example
Answer: No. Retain the negative accuracy result, but real human multi-dimensional preferences and open generation judging are absent. Track per-dimension mean, variance and effective weights.
Illustrative example: Brevity can improve while exact match worsens, so answer score alone isn't complete.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
How does GPRL differ from summing rewards then applying GRPO?
Show answer and example
Answer: Summation mixes scales and conflicts before normalization. Separate standardization preserves each dimension's relative information before aggregation; the operations do not commute. Test context weights and drift separately.
Illustrative example: One high-variance dimension can dominate normalized totals while others barely matter.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
Why compute GPRL group advantages per preference dimension?
Show answer and example
Answer: Dimensions have different units and variance. Normalize separately, then aggregate by task context. Normalization avoids accidental numerical dominance, not a claim of equal importance.
Illustrative example: A 100-point format scale would swamp binary correctness under raw summation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Core mechanism
Why ground sharding proposals in profiling rather than retrieving a common best configuration?
PROMPTS: Performance Optimization via Multi-Agent Planning for LLM Training and Serving
Show answer and example
Answer: Optimal parallelism depends on measured compute, communication, and memory constraints. The knowledge base offers options; Analyzer locates bottlenecks, Proposal generates explainable candidates, and Coordinator organizes verification. A config or formula check does not run training or measure speedup.
Illustrative example: Example: A configuration is fast on a model card, but local profiling shows all-reduce dominates. Compare options that address communication.
Sections in the Chinese source: 背景与主要改动, 核心公式或操作. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Implementation details
Do local profiler ranking and bounded proposals include TPU/GSPMD measurements?
PROMPTS: Performance Optimization via Multi-Agent Planning for LLM Training and Serving
Show answer and example
Answer: No. Local reference tests cover evidence ranking and proposal constraints without Google TPU profiling, GSPMD, or a production workload matrix. Top-three hits and peak gains are author measurements, not consequences of local candidate ordering.
Illustrative example: Example: Synthetic profiling can recommend less communication. Until compiling and running that configuration, there is no observed training speed.
Sections in the Chinese source: 核心公式或操作, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Method comparison
How does separating Coordinator, Analyzer, and Proposal differ from configuration-table search?
PROMPTS: Performance Optimization via Multi-Agent Planning for LLM Training and Serving
Show answer and example
Answer: Separate roles coordinate work, explain bottlenecks, and propose targeted settings from evidence. Table search mainly enumerates predefined options. Multiple agents have their own cost; real training or serving results determine benefit, not explanation length or votes.
Illustrative example: Example: Analyzer identifies communication as dominant, then Proposal changes sharding accordingly instead of testing every dimension randomly.
Sections in the Chinese source: 背景与主要改动, 核心公式或操作. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Basics
Why does PROMPTS treat model-parallel partitioning as iterative measured optimization?
PROMPTS: Performance Optimization via Multi-Agent Planning for LLM Training and Serving
Show answer and example
Answer: Training bottlenecks vary by hardware and configuration among compute, memory, and communication. Agents use profiler evidence to propose partitioning and observe effects rather than apply stock settings. A recommendation is not a speedup until measured for runtime, resources, and feasibility on target hardware.
Illustrative example: Example: More sharding may save memory but slow cross-device communication. Profiling distinguishes the bottleneck.
Sections in the Chinese source: 背景与主要改动, 核心公式或操作. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Core mechanism
Why maintain a policy-controlled taxonomy before small-model retrieval and single-token scoring in Policy Facet?
Policy-Grounded Dynamic Facet Suggestions for Job Search
Show answer and example
Answer: A curated set bounds themes, preventing arbitrary out-of-policy generation. Embeddings retrieve cheaply within it and a compact generator scores a token such as 'Okay' conditioned on user and query. Caching and batching lower latency. High model probability does not complete policy enforcement.
Illustrative example: Illustration: Choose a family travel facet from maintained destination and activity categories rather than inventing one live.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Implementation details
Why verify tokenizer and full conditioning prefix for Policy Facet's 'Okay' score?
Policy-Grounded Dynamic Facet Suggestions for Job Search
Show answer and example
Answer: The word may tokenize differently across models. Read the designated output under the right user, query, and facet prefix. A hardcoded vocabulary ID can score another word after model changes; caches must not include another facet. Local proxy latency is not production evidence.
Illustrative example: Illustration: If 'Okay' becomes two tokens, the old single ID may be wrong even though it returns a finite number.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Method comparison
Why is an 'Okay' token probability cheaper than generating full judgments?
Policy-Grounded Dynamic Facet Suggestions for Job Search
Show answer and example
Answer: A bounded output gives a pointwise score without long decoding for every candidate; prefix caching and batching add reuse. It loses detailed reasoning and needs calibration. Scoring cannot recover facets missed in retrieval.
Illustrative example: Illustration: Read one probability for ten facets instead of generating ten long reviews, but an unretrieved option never gets scored.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Basics
Are Policy Facet's facets the same candidates as final products or documents?
Policy-Grounded Dynamic Facet Suggestions for Job Search
Show answer and example
Answer: No. Facets are refinement options that alter later search or browsing; items satisfy the resulting request. A facet must meet taxonomy and usefulness requirements. A high-scoring item title does not automatically make a useful refinement.
Illustrative example: Illustration: 'Family activities' narrows travel search; a specific museum ticket is a result.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Core mechanism
Why include general replay in enterprise code adaptation?
Customizing an LLM for Enterprise Software Engineering
Show answer and example
Answer: Internal examples teach private code, tools, and workflows but can displace general ability. Replay adds a retention constraint; deduplication, filtering, and contamination controls protect the comparison. Validate both sides.
Illustrative example: Learning internal build commands need not preserve general reasoning if training drops all general examples.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Implementation details
What do the two extreme replay weights verify?
Customizing an LLM for Enterprise Software Engineering
Show answer and example
Answer: λ=0 leaves enterprise loss and λ=1 leaves general replay. This checks branch wiring and coefficient direction. Local tests use no Google private data, Gemini weights, or A/B.
Illustrative example: Enterprise loss dominating at λ=1 suggests reversed weighting or routing.
Sections in the Chinese source: 核心公式, 复现边界. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Method comparison
What extra constraints distinguish enterprise midtraining?
Customizing an LLM for Enterprise Software Engineering
Show answer and example
Answer: It includes internal code, interaction, and tools while controlling duplicates and evaluation leakage and preserving general ability. General training loss does not replace internal workflow evaluation.
Illustrative example: Memorizing old fixes can lower loss without helping new internal problems.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Basics
Why not continue training on all internal code unchanged?
Customizing an LLM for Enterprise Software Engineering
Show answer and example
Answer: Extract high-value history and interactions, deduplicate, filter, and control contamination before midtraining and internal-tool post-training. Data curation and evaluation are integral.
Illustrative example: Generated duplicate files and valuable fixes should not receive equal treatment by default.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
What does CQ-SID gain and lose by fixing the first code to the item category?
Efficient Generative Retrieval for E-commerce Search with Semantic Cluster IDs and Expert-Guided RL
Show answer and example
Answer: The category organizes coarse search and business constraints; residual codes encode within-category detail. Wrong categories can make candidates unreachable, and cross-category substitutes may be split. Expert-guided reward balances clicks, purchases, and constraints; taxonomy alone does not ensure relevance.
Illustrative example: Generate a shoe code before styles, but a multifunctional product wrongly classified as luggage may never appear in a shoe search.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Within what scope should CQ-SID normalize grouped advantages?
Efficient Generative Retrieval for E-commerce Search with Semantic Cluster IDs and Expert-Guided RL
Show answer and example
Answer: Compute reward mean and standard deviation among rollouts for the same context and use the defined epsilon near zero variance. Mixing requests confounds different reward difficulties. Category and residual codes must share the legal vocabulary interface.
Illustrative example: Five high rewards for an easy request must not lower all advantages for a hard request; compare candidates within their own request.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
Where does CQ-SID's category constraint differ from a freely learned RQ first code?
Efficient Generative Retrieval for E-commerce Search with Semantic Cluster IDs and Expert-Guided RL
Show answer and example
Answer: Free quantization groups coarse codes by representation distance. CQ-SID fixes alignment with taxonomy in training and decoding, reducing semantic drift but depending on catalog quality. Expert-Guided RL separately aligns feedback; clear categories do not ensure higher recall.
Illustrative example: A photography book and tutorial video may share a free prefix but split under a fixed catalog taxonomy.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Why use a category first and within-category residuals afterward in CQ-SID?
Efficient Generative Retrieval for E-commerce Search with Semantic Cluster IDs and Expert-Guided RL
Show answer and example
Answer: Categories give controllable semantic boundaries; later codes distinguish details inside them. This aligns decoding with catalog constraints but may block items under a wrong category or separate cross-category neighbors.
Illustrative example: Choose camera first, then distinguish mirrorless from action cameras. A product misfiled under phone accessories follows the wrong path.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
Why use length-normalized sequence scores for InfoNCE rather than cumulative token reward?
Show answer and example
Answer: Summed logp changes scale with length and can confuse verbosity with quality. Average per response then contrast within the same prompt. Temperature and normalization still affect gradients; relative quality is not absolute correctness.
Illustrative example: Two correct replies can differ in length; average before comparing so raw sum does not penalize the longer one.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
Where do length normalization and temperature apply in ConSPO?
Show answer and example
Answer: Average valid-response logp, divide by temperature, then use stable group log-sum-exp. Exclude prompts/padding. Local kernel is not paper-scale training.
Illustrative example: Changing tokenwise temperature and denominator arbitrarily departs from sequence scoring.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
How does ConSPO group contrast differ from independent binary pairs?
Show answer and example
Answer: One denominator couples all candidates, unlike independent pair losses. Group size and negative composition change the objective.
Illustrative example: A new competitor changes the shared denominator, not just one pair.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
Why use group InfoNCE for sequence preference in ConSPO?
Show answer and example
Answer: Multiple same-prompt sequences compete in one objective. Average logp reduces length scale effects and temperature controls separation.
Illustrative example: Boost a good solution relative to alternatives rather than score its likelihood alone.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Core mechanism
Why isn't the principle just sequencing GRPO and KL functions?
Show answer and example
Answer: The paper tests four stages under equal label budgets and distinguishes forward KL on teacher-visited states from reverse KL on student states. Operators alone don't reproduce it. No local complete training.
Illustrative example: The same KL name on a teacher solution and a student error refers to different supervision.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Implementation details
Do existing GRPO and OPD operators reproduce the four stages?
Show answer and example
Answer: No. Literature only; no full scheduling, real frozen teacher or equal-budget runs. Shared tokenizer and other prerequisites still matter. Stage labels aren't execution.
Illustrative example: Listing four configuration stages does not validate their data flow.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Method comparison
Is the same KL direction equivalent on teacher and student occupancy?
Show answer and example
Answer: No. Teacher ideal and student actual prefixes visit different errors. State source and divergence together define supervision.
Illustrative example: A KL on a correct proof and on a wrong intermediate state teaches different recovery.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Basics
Why emphasize equal-label-budget stages in the sparse-to-dense reward principle?
Show answer and example
Answer: Dense teachers have different supervision costs. Match label budgets and stages to understand ordering. Contribution is an empirical four-stage comparison, not another KL function.
Illustrative example: More teacher calls can improve scores without proving dense objectives caused the gain.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
Why can't the total ratio disentangle mismatch and staleness without old training logits?
Show answer and example
Answer: Current training vs old inference factors through old training probabilities. Missing that intermediate hides whether changes come from the engine or updates. Recover the anchor by snapshot or an old-logit model, not zero-fill.
Illustrative example: A changed score on an old sample cannot identify the original engine gap without its training snapshot.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
Can current logits simply replace missing old training logits?
Show answer and example
Answer: No. Snapshots, old-logit models, interrupted synchronization and EWMA are explicit alternatives with costs/assumptions. Local decomposition and state tests are not cluster deployment.
Illustrative example: Rescoring old samples with updated weights rewrites the baseline rather than restoring history.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
Why do train–inference and staleness corrections need different denominators?
Show answer and example
Answer: One compares training and serving implementations of the same old policy; the other compares current and old training policies. Their product is the total ratio, not interchangeable terms.
Illustrative example: Changing inference engines at fixed weights can create mismatch without training staleness.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
Why is Missing Old Logits more than a logging issue?
Show answer and example
Answer: Historical training-side probabilities separate train–inference mismatch from stale weights. Serving probabilities alone conflate both, making the correction ambiguous.
Illustrative example: Token drift can come from engine differences or several policy updates.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why turn visual content into normalized semantic tokens in MM-LLM instead of running a large model per ranking request?
Show answer and example
Answer: Visual vectors are rich but fine-grained semantics can be hard to use consistently. The offline visual encoder, Q-Former, and LM generate captions normalized into reusable category tokens. A smaller online ranker combines them with IDs and visual features, amortizing semantic compute.
Illustrative example: Illustration: Extract rain, camping, and gear tutorial tokens offline from a thumbnail, then rank using those features and history without asking the LM to explain the image again.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
Why are MM-LLM's four local caption tokens not a full visual–LLaMA2 training reproduction?
Show answer and example
Answer: Public content vectors replace private images, and light query cross-attention generates proxy tokens. This tests reduced fusion without the original visual encoder, Q-Former, or full large model. Report internal ID+content and DIN comparisons separately.
Illustrative example: Illustration: Four tokens made from genres do not demonstrate image understanding, even if fusion loss falls.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
What can MM-LLM caption tokens add beyond visual embeddings?
Show answer and example
Answer: Continuous visual representations retain perceptual structure; captions extract language-level attributes and uses. They can duplicate or contradict each other, motivating gated rather than blind fusion. The local caption ablation was near flat and Hit fell, so readability is not evidence of ranking gains.
Illustrative example: Illustration: The visual vector reflects color and layout; a caption can identify a waterproofing demo or add noise if it contributes no new concept.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why normalize MM-LLM captions into categorical token IDs?
Show answer and example
Answer: Free-form descriptions vary in wording and length. Category IDs provide a stable cacheable interface for ID, visual, and history-profile fusion. Normalization loses detail and inherits caption errors, so discrete IDs are not ground-truth annotations.
Illustrative example: Illustration: 'Camping in rain' and 'putting up a tent in the rain' can map to similar tokens; a false weather caption produces a false tag.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why share memory by first-level target SID instead of per-item or all-item compression?
UxSID: Semantic-Aware User Interests Modeling for Ultra-Long Sequence
Show answer and example
Answer: Grouping amortizes full history while retaining target sensitivity. Global SID response gates anchor probing; independent PFFNs and orthogonality mitigate collapse. Within-group details can still be flattened.
Illustrative example: Camping items share one camping memory while cameras use another.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
What details matter for per-anchor PFFN and orthogonality?
UxSID: Semantic-Aware User Interests Modeling for Ultra-Long Sequence
Show answer and example
Answer: Give anchors independent PFFNs and normalize orthogonality by the whole matrix's squared L2 norm, not row cosine norms. Local two-level k-means/residual quantization is not LETTER.
Illustrative example: Sharing one FFN changes anchor specialization even at matching shapes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
Why are global and local probes not duplicate attention?
UxSID: Semantic-Aware User Interests Modeling for Ultra-Long Sequence
Show answer and example
Answer: Query full history for a global response, gate the target SID, then read compressed anchors. The inputs differ; anchor-only loses global modulation.
Illustrative example: Determine overall category interest before inspecting individual anchors.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why share UxSID memory by user and target SID?
UxSID: Semantic-Aware User Interests Modeling for Ultra-Long Sequence
Show answer and example
Answer: Per-item retrieval is expensive and one user-only summary is too coarse. Similar first-level SID targets share memory, trading cache cost for targeting without assuming identical preferences.
Illustrative example: Several shoes can reuse interest memory while retaining item differences.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why preselect PRL-PUTS weights via two Q heads and a Pareto sweep?
Show answer and example
Answer: Each head estimates a business utility and weight mixtures choose trade-offs. An offline sweep drops dominated strategies and yields a governable frontier; serving can switch weights without swapping models. Q estimation error and drift make offline non-dominance insufficient for online superiority.
Illustrative example: Illustration: Keep one policy strong in saves and another strong in deeper interactions, and discard one worse in both rather than testing arbitrary weights on traffic.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Why hold Q model and action-space versions fixed when switching PRL-PUTS weights?
Show answer and example
Answer: Weight effects are identifiable only over the same comparable Q values and action set. Keep α in range and record each head's units and reward definition. A fixed local scorer does not reproduce full long-term Q-learning and online governance.
Illustrative example: Illustration: Switching 70/30 to 50/50 while retraining a Q head confounds attribution.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
What choices does Pareto sweeping retain beyond one highest weighted PRL-PUTS score?
Show answer and example
Answer: One weighting selects one compromise. Sweeping and removing dominated strategies retains options for changing goals. Offline noise can create false frontier points, so uncertainty and online tests matter.
Illustrative example: Illustration: Keep stronger-save and stronger-engagement policies, but reject one worse on both rather than promoting it for an arbitrary score.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Does Pareto non-dominance in PRL-PUTS require improvement on all objectives?
Show answer and example
Answer: No. A policy remains when none is at least as good on every goal and better on at least one. Different Repin/P2P trade-offs can be nondominated. Suitability still depends on business needs, constraints, and estimation error.
Illustrative example: Illustration: (8,3) and (6,5) trade off while (5,2) is dominated. The first two are not necessarily better than production.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why combine commercial SID, value logits, and request-specific tries?
Unified Value Alignment for Generative Recommendation in Industrial Advertising
Show answer and example
Answer: Commercial IDs carry value features; fused logits and eCPM reward shape policy. The trie separately enforces current eligibility. High value does not mean legal or relevant; local value is a proxy.
Illustrative example: Block a high-bid but unavailable ad and score legal alternatives by both relevance and value.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
How distinguish local UniVA from real eCPM and personalized serving tries?
Unified Value Alignment for Generative Recommendation in Industrial Advertising
Show answer and example
Answer: Local network/RL/trie paths use deterministic bid/business proxies. Limit requests to currently eligible inventory and accumulate fused log probability; a globally valid ad may be illegal here.
Illustrative example: Mask an ad not eligible in the current region.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
How does Generation-as-Ranking differ from independent post-generation ranking?
Unified Value Alignment for Generative Recommendation in Industrial Advertising
Show answer and example
Answer: Generation and value logits sum before normalization, so value changes every path probability. Post-ranking only reorders retrieved candidates.
Illustrative example: A high-value path can enter the beam earlier rather than be considered after recall.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Does the final commercial SID level just refine item semantics?
Unified Value Alignment for Generative Recommendation in Industrial Advertising
Show answer and example
Answer: No. It keeps two semantic levels and adds commercial attributes plus within-class bid bins. Interpret the commercial mapping rather than ordinary content clusters; proxy bids are not revenue.
Illustrative example: Similar ads can differ at the final level because of bid tier.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Core mechanism
Why align passive browsing to active queries before discretization and reweighting in PA-Bridge?
Show answer and example
Answer: Browsing says what was seen, while active queries express what users want to ask. Direct reuse can suggest familiar but poor conversation starters. Distribution alignment bridges intent spaces, and discrete units with reweighting reduce repeated dominance by popular exposures.
Illustrative example: Illustration: Frequent fitness videos may coexist with a specific need to ask about training with knee discomfort; query alignment can surface the latter.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Implementation details
What temporal constraint applies when fitting PA-Bridge hpassive to hactive pairs?
Show answer and example
Answer: Use active queries as training targets and cut passive context before the query. Codes and weights must also use allowed history. The public proxy lacks genuine active-query logs, so an Align function alone does not demonstrate cross-behavior transfer.
Illustrative example: Illustration: Rehab videos watched after asking about a knee cannot be part of the prefix used to predict that question.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Method comparison
How does PA-Bridge reweighting differ from selecting the most-browsed topic?
Show answer and example
Answer: Counts reflect prior exposure, so top frequency reinforces what was already shown. Reweighting reduces dominant semantic units and broadens active topics, but too much favors irrelevant tail content. Diversity is not satisfaction or causal debiasing.
Illustrative example: Illustration: Reweighting can surface related strength training alongside common weight-loss content, not justify arbitrary unrelated topics.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Basics
Why isn't PA-Bridge passive-to-active transfer just rewriting browsed titles as questions?
Show answer and example
Answer: Active queries express actions and problems absent from passive views. Align distributions, then use semantic codes and weights to reduce exposure loops. A question mark cannot create user intent, and model inference is not an explicit request.
Illustrative example: Illustration: 'How do I exercise?' rewrites a title but may miss the actual question about protecting a knee.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Core mechanism
Why does EKTM transfer CTR features by task similarity rather than sharing equally across conversions?
Effective Knowledge Transfer for Multi-Task Recommendation Models
Show answer and example
Answer: Click labels are plentiful but relate differently to each conversion. Similarity controls transfer strength, hard-example supervision reduces negative transfer, and CVR towers retain their own targets. Validate similarities; abundant CTR data are not universally trustworthy.
Illustrative example: Cart additions may relate closely to clicks while long-term renewal depends on other factors, so they should not inherit the same strength.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Implementation details
Why verify task identity, not only shape, when EKTM combines fctr and weighted CVR features?
Effective Knowledge Transfer for Multi-Task Recommendation Models
Show answer and example
Answer: Each similarity weight is tied to a source-target task and vectors must be semantically aligned. Preserve ordering mappings and derive hard-example masks from valid labels, never test scores. Public proxies do not create missing production conversion labels.
Illustrative example: A CTR-to-purchase weight can multiply a favorites tower without a shape error, yet corrupt supervision.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Method comparison
Where does EKTM control conflicts compared with a shared-bottom model?
Effective Knowledge Transfer for Multi-Task Recommendation Models
Show answer and example
Answer: Shared-bottom uses the same backbone for all tasks. EKTM weights task-specific contributions during transfer and constrains harmful sharing with hard examples. This adds conditional specialization, similarity dependence, and tower cost; a movie proxy does not prove conversion improvement.
Illustrative example: A purchase tower can borrow more from add-to-cart knowledge than an unrelated tower instead of competing for an identical shared representation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Basics
Why transfer CTR knowledge into multiple EKTM CVR tasks instead of learning each from scratch?
Effective Knowledge Transfer for Multi-Task Recommendation Models
Show answer and example
Answer: Dense clicks can give relevance and interest knowledge to sparse conversions, but tasks differ. Similarity controls borrowing and hard-example loss limits harmful transfer. Measure each target separately.
Illustrative example: Cart and purchase can share interest signals, while returns mean something different; both zero transfer and indiscriminate transfer have risks.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Core mechanism
Why filter generated expansion terms using document frequency in the inverted index?
Superintelligent Retrieval Agent: The Next Frontier of Agentic Retrieval
Show answer and example
Answer: An LLM can suggest plausible terms absent from the corpus or ubiquitous terms that cannot discriminate. Document-frequency constraints ensure terms are retrievable and not too broad. One weighted BM25 query then combines them with the original. It preserves lexical retrieval rather than treating imagined terms as evidence.
Illustrative example: Example: An abbreviation absent from the corpus cannot retrieve documents about a rare disease, while research is too broad. Filtering retains useful terms.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Implementation details
How should zero document frequency, the upper bound, and original query weight be handled?
Superintelligent Retrieval Agent: The Next Frontier of Agentic Retrieval
Show answer and example
Answer: An expansion term must satisfy 0<DF(t)≤τ|C|. Exclude zero-frequency terms and those above the bound. Preserve original-query BM25 and add the expansion contribution multiplied by w. Do not replace the original query or silently introduce multiple retrieval rounds. The local run implements the gate and combination, not the paper-scale index and LLM.
Illustrative example: Example: If no expansion term survives filtering, still return results for the original query rather than dropping the whole query.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Method comparison
How does a single weighted BM25 expansion differ from multi-turn agent search?
Superintelligent Retrieval Agent: The Next Frontier of Agentic Retrieval
Show answer and example
Answer: SIRA generates and filters missing terms, then runs one weighted retrieval with the original and expanded terms. It does not iteratively read results and replan. This simplifies interaction but limits recovery from first-pass failure. An agent label does not imply multi-turn self-correction.
Illustrative example: Example: After generating three terms, the method returns one ranking. It does not read first-round pages and invent another set of questions unless a separate loop is added.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Basics
Why does SIRA focus on vocabulary mismatch between how users ask and how evidence is written?
Superintelligent Retrieval Agent: The Next Frontier of Agentic Retrieval
Show answer and example
Answer: Inverted retrieval relies on term overlap. Users describe phenomena in ordinary language while documents may use technical terms, hiding relevant evidence. SIRA uses a frozen LLM to supply possible evidence terms and constrains expansion to the corpus. It bridges query–document vocabulary, not trains an answer generator.
Illustrative example: Example: A user says a program uses more memory over time. Documentation might call it a memory leak. Adding that term can retrieve otherwise missed material.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why regenerate RecGPT-Mobile queries only when joint drift passes a threshold?
Show answer and example
Answer: On-device inference still costs battery and time. Entropy, overlap, and divergence test meaningful intent changes, and a prompt budget limits per-call cost. The device converts behavior to intent and the server retrieves; gating may miss subtle important shifts.
Illustrative example: Illustration: Repeated shoe views need not trigger repeatedly, while switching to baby products can trigger a new server query.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
How verify RecGPT-Mobile's prompt budget and drift trigger separately?
Show answer and example
Answer: Select components by marginal gain within a token cap, and trigger by entropy, Jaccard, and JS changes. These govern per-call cost and call frequency. Future test queries cannot choose either. Small-sample semantic accuracy is not device interaction success.
Illustrative example: Illustration: A needed refresh does not permit an unlimited prompt; even a short prompt need not be recomputed if intent is stable.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
Why are INT8 storage reduction and INT8 inference speed distinct evidence?
Show answer and example
Answer: Eight-bit serialization reduces file size; dequantizing to float kernels can leave runtime unchanged or slower. True quantized compute needs supported hardware and operators. The Mac did not enable INT8 linear, so half storage is not half latency.
Illustrative example: Illustration: A compressed file saves transfer while unpacking and generation still take time.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why run behavior-to-query on-device but keep RecGPT-Mobile retrieval server-side?
Show answer and example
Answer: Local context can generate a query with a small quantized model without sending raw history. The server maintains the large catalog. Devices still pay energy, compute, and updates. Privacy depends on actual transmission, not the label on-device.
Illustrative example: Illustration: Send a camping-light query, not the browsing log that generated it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why compress both SID vocabulary and temporal sequence?
Beyond Item IDs: Scaling Short-Form-Video Recommendation via Semantic-Native Long Sequence Modeling
Show answer and example
Answer: Atomic ID tables grow with inventory, and full attention grows quadratically. Bounded hierarchical SID handles representation size; event folding reduces time positions and global queries gather interests. Both sacrifice detail.
Illustrative example: Fold 48 events into 12 local tokens and share semantic codes, but do not assume original within-window order survives.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Does the local folding exactly match paper channel rearrangement?
Beyond Item IDs: Scaling Short-Form-Video Recommendation via Semantic-Native Long Sequence Modeling
Show answer and example
Answer: No. Mean/recent is a proxy; no Google RQ-VAE or production components. Pool always-valid global queries and exclude padding.
Illustrative example: Mean folding tests coarse compression, not the paper tensor layout.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does temporal folding differ from retaining only recent history?
Beyond Item IDs: Scaling Short-Form-Video Recommendation via Semantic-Native Long Sequence Modeling
Show answer and example
Answer: Folding combines neighboring events into features and keeps coarse older information; truncation drops it. Folding and global queries do not preserve full temporal resolution.
Illustrative example: Four events compressed into one token can lose ordering without discarding all evidence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Does a shared Semantic-Native SID prefix mean identical videos?
Beyond Item IDs: Scaling Short-Form-Video Recommendation via Semantic-Native Long Sequence Modeling
Show answer and example
Answer: No. Prefixes encode coarse similarity; identity still depends on the complete path and catalog mapping. Semantic sharing does not merge interaction records.
Illustrative example: Distinct camping videos can share a prefix but keep different logs.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Core mechanism
Why require less separation for semantically similar answers?
Mind the Gap: Structure-Aware Consistency in Preference Learning
Show answer and example
Answer: A large fixed margin spends capacity separating paraphrases and amplifies label noise. SA-DPO scales the added margin with frozen semantic distance, zero at zero distance, while keeping relative-to-reference preference. Control encoder quality and mean pressure separately.
Illustrative example: 'The answer is three' and 'The result is 3' need less separation than right vs wrong answers; wording can still bias distances.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Implementation details
What should SA-DPO do when response distance is zero?
Mind the Gap: Structure-Aware Consistency in Preference Learning
Show answer and example
Answer: Set extra distance margin to zero, but retain the policy/reference log-ratio difference in preference loss. Use frozen distances, not constants or test-label calibration.
Illustrative example: Removing extra separation for identical wording does not make the whole loss zero.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Method comparison
Why match mean margin when comparing SA-DPO to a fixed-margin method?
Mind the Gap: Structure-Aware Consistency in Preference Learning
Show answer and example
Answer: Otherwise performance can change from overall training pressure rather than semantic allocation. Matching means better isolates allocation without proving semantic judgments always correct.
Illustrative example: An easier objective from a smaller mean margin does not establish better semantics.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Basics
Does SA-DPO measure semantic distance between responses or between prompts?
Mind the Gap: Structure-Aware Consistency in Preference Learning
Show answer and example
Answer: It compares preferred and dispreferred answers to one prompt. It is not a margin from prompt difficulty/length. Frozen response representations supply semantic distances independently of preference labels; distances aren't correctness scores.
Illustrative example: Two paraphrases of one answer can be close; two similar prompts are not the comparison unit.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Core mechanism
Why vary expert margins rather than add minority class weights?
Optimized Deferral for Imbalanced Settings
Show answer and example
Answer: Margins change score separation geometry; weights scale loss contributions. mild retains full training costs or rewards instead of reducing each to a winner. Estimate margins on training and never read unknown expert costs at inference.
Illustrative example: Hard minimum-cost labels can discard near-ties involving a specialist.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Implementation details
May mild route using true candidate costs at inference?
Optimized Deferral for Imbalanced Settings
Show answer and example
Answer: No. Routing uses inference features only; costs and labels are unavailable. Fit margins on train and select on validation. Use stable log-sum-exp and choose the highest expert index on ties as specified.
Illustrative example: Training knows answer costs; a new question does not reveal who answers correctly.
Sections in the Chinese source: 核心公式, 代码映射与边界. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Method comparison
What does full expert cost/reward supervision retain over argmin labels?
Optimized Deferral for Imbalanced Settings
Show answer and example
Answer: Argmin keeps only the winner. A cost-based surrogate and expert margins preserve differences and relative score constraints.
Illustrative example: A close second and an expensive second need not give identical supervision.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Basics
Why can imbalance hurt rare specialist experts?
Optimized Deferral for Imbalanced Settings
Show answer and example
Answer: Broad experts can dominate router learning despite specialists serving particular inputs. Expert-specific margins and full training reward structure retain more information than hard minimum-cost labels.
Illustrative example: Majority-oriented training may ignore a specialist for a rare question type.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Core mechanism
Why does GloRank move beyond ordinary reranking when it replaces local IDs with global SIDs?
From Local Indices to Global Identifiers: Generative Reranking via Global Action Space
Show answer and example
Answer: Local IDs only address upstream candidates. Global SIDs allow a broader catalog, so training learns nomination as well as list utility. Larger action spaces complicate search and legality and require evaluation different from closed-pool reranking.
Illustrative example: Index 11 is invalid in a ten-item local list; a global code can nominate a new item only if it maps to a serveable catalog entry.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Why can't request-local indexes replace global SIDs in GloRank listwise SFT and group rewards?
From Local Indices to Global Identifiers: Generative Reranking via Global Action Space
Show answer and example
Answer: Local indexes change with candidate order and cannot identify catalog items consistently. Train and decode mapped global paths, and score actual generated identities without fixing invalid codes using test answers. Do not claim full global language training from a proxy.
Illustrative example: Index three can mean camera in one request and tent in another, so it is not a stable global token.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
Why state candidate access when comparing global generation and local reranking?
From Local Indices to Global Identifiers: Generative Reranking via Global Action Space
Show answer and example
Answer: Full-catalog generation can improve coverage independently of ordering. Define full or local evaluation, legal mappings, and reward protocols. Different access is not a single-component ablation, and preserve local negative results.
Illustrative example: Finding the right book outside a fixed pool does not show a better reranker; the local model could not choose it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why can GloRank select items outside the local candidate pool with global SIDs?
From Local Indices to Global Identifiers: Generative Reranking via Global Action Space
Show answer and example
Answer: Global identifiers define actions over the catalog instead of the current list, adding nomination to reranking. This expands coverage while increasing search, legal mapping, and list-objective difficulty. Fixed-candidate assumptions no longer describe all results.
Illustrative example: A niche book omitted from five popular local candidates can be nominated globally if legal and relevant.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why not penalize all same-SID pairs equally?
Show answer and example
Answer: Some overlap is useful semantics, some harmful congestion. AdaSID adapts to load, compatibility, and training phase. Uniform penalties can destroy meaningful sharing or tolerate overloaded collisions.
Illustrative example: Similar running shoes can share a coarse code; shoes and ovens sharing a crowded code may need stronger correction.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
Is AdaSID's collision coefficient a single scalar?
Show answer and example
Answer: λij(t) depends on pair load, compatibility, and time. Apply it to the corresponding overlap before summing and report the schedule. A constant removes adaptation.
Illustrative example: Shoes and books can receive different penalties at the same step and change later.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
What does an adaptive coefficient change compared with fixed collision loss?
Show answer and example
Answer: Pair and training time affect severity, allowing early semantic organization and later discrimination. Flexibility adds calibration cost and still needs matched evaluation.
Illustrative example: Related books can share a code early, with high-load distinctions strengthened later.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why isn't SID overlap always harmful?
Show answer and example
Answer: Semantic sharing transfers related information; harmful collisions obstruct discrimination or overload learning. Adapt penalties rather than maximize separation everywhere.
Illustrative example: Color variants can share coarse clothing semantics while still differing at fine inventory codes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Core mechanism
Why select a query before reading target scores and exclude the target from history?
Show answer and example
Answer: Historical correlations supply a prior. Only actively selected responses update the posterior. An entire target column in covariance or selection leaks unqueried answers. Variance reduction does not guarantee robustness beyond historical distribution.
Illustrative example: Select the most informative item first and query its score, rather than preload the target's full column.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Implementation details
Why reject duplicate target observations and target IDs in history?
Show answer and example
Answer: Duplicates distort query budget and posterior updates; historical target scores leak unqueried outcomes into the prior. Only selected feedback is allowed. Three history records are a local minimum, not a theorem guarantee.
Illustrative example: A target column already present in history invalidates claimed savings.
Sections in the Chinese source: 核心公式, 代码映射与边界. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Method comparison
How does Bayesian quadrature acquisition differ from choosing the lowest current mean?
Show answer and example
Answer: BQ maximizes expected reduction of overall wᵀKw variance, not difficulty. Failure-oriented acquisition has separate candidate conditions.
Illustrative example: An easy item correlated with many unqueried items can reduce score uncertainty substantially.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Basics
Why reuse historical per-item scores?
Show answer and example
Answer: Cross-model correlations form a probabilistic surrogate, updated after limited queries to estimate total score and uncertainty. Estimates are not full evaluations.
Illustrative example: Query several items and predict the rest while reporting posterior variance.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Core mechanism
Why can't CS3 write candidate vectors back into all shared user representations to strengthen interaction?
CS3: Efficient Online Capability Synergy for Two-Tower Recommendation
Show answer and example
Answer: Efficient two-tower retrieval needs independently computed user and item vectors. Candidate-specific user states cannot retrieve the full catalog with one query. CS3 uses deployable cyclic refinement and knowledge collaboration; track which updates are offline-materializable and which depend on candidates. Training interactions are not free ANN features.
Illustrative example: Adding item A's details can improve scoring A but makes the vector unsuitable as the same query for B and C.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Implementation details
How should CS3 cyclic refinement distinguish offline item state from online candidate-dependent state?
CS3: Efficient Online Capability Synergy for Two-Tower Recommendation
Show answer and example
Answer: Trace whether an update reads the current user or candidate; downstream tensors are no longer global cache values. Match gate and residual dimensions and specify previous versus current iteration state, avoiding accidental in-place semantics. Preserve the actual cycle, cross-sync, and loss paths.
Illustrative example: Refining item i for user A must not overwrite the generic vector read by user B.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Method comparison
How does CS3 gated residual refinement differ from replacing the two towers with a cross-encoder?
CS3: Efficient Online Capability Synergy for Two-Tower Recommendation
Show answer and example
Answer: Gated residuals make controlled updates around the two-tower state and use collaborative losses while retaining the serving interface. Cross-encoders jointly encode each pair. Their compute and index dependencies differ; the matched local result was negative, so quality preservation is not established.
Illustrative example: A bounded correction to a retrievable vector is cheaper and structurally different from re-encoding every user-item pair from scratch.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Basics
Why is interaction in CS3 a trade-off between retrieval efficiency and expressivity?
CS3: Efficient Online Capability Synergy for Two-Tower Recommendation
Show answer and example
Answer: Independent towers enable precomputed item vectors and one user vector for nearest-neighbor retrieval. Deep interactions usually depend on pairs and cannot be globally cached. Cyclic refinement and knowledge collaboration add links while respecting deployment boundaries.
Illustrative example: If each user's item vector differed, a single shared index could not serve it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Core mechanism
Why require Critic, SkillHub, and real feedback after an Actor proposes a winner?
AgenticRecTune: Multi-Agent with Self-Evolving Skillhub for Recommendation System Optimization
Show answer and example
Answer: Proposals are hypotheses. Critic checks feasibility, runs supply evidence, and Insight/Skill record conditions and failures. Self-ratings are not results. Local search is bounded and selection must isolate test.
Illustrative example: Record a diverse but over-budget configuration so future rounds avoid repeating it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Implementation details
How prevent test tuning across three local rounds?
AgenticRecTune: Multi-Agent with Self-Evolving Skillhub for Recommendation System Optimization
Show answer and example
Answer: Actor proposes bounded genomes and Critic selects only on validation, then saves evidence. Run test once after final selection. No real online authority is involved.
Illustrative example: Do not switch winners after seeing a disappointing final test score.
Sections in the Chinese source: 背景与主要改动, 本地复现. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Method comparison
How does SkillHub differ from restarting each round with the same prompt?
AgenticRecTune: Multi-Agent with Self-Evolving Skillhub for Recommendation System Optimization
Show answer and example
Answer: It retains auditable feedback-backed constraints and strategies, reducing repeated failures. One high score is not automatically reliable and transfer depends on context.
Illustrative example: Carry forward the finding that one diversity setting hurts new users.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Basics
Why search cross-stage recipes instead of one learning rate?
AgenticRecTune: Multi-Agent with Self-Evolving Skillhub for Recommendation System Optimization
Show answer and example
Answer: Retrieval, fusion, and reranking shape each other's distributions. Agents propose and review quality/diversity/risk across the recipe, with evidence for each stage and end-to-end effects.
Illustrative example: Diversifying ranker scores cannot help if retrieval never provides varied categories.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why use permutation grouping and global tokens against RankUp rank collapse instead of merely adding embeddings or depth?
RankUp: Towards High-rank Representations for Large Scale Advertising Recommender Systems
Show answer and example
Answer: Added parameters can reuse the same subspace. Different groupings and tables alter interaction paths, global tokens connect groups, and external features and task decoupling add signals. These aim to turn capacity into richer directions, not guarantee effective rank.
Illustrative example: Illustration: New groupings let fields interact that never shared a group under a fixed partition.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why track RankUp permutation matrices and field identities together?
RankUp: Towards High-rank Representations for Large Scale Advertising Recommender Systems
Show answer and example
Answer: XP rearranges inputs; corresponding embedding and group weights must follow the same map. Align global-token residual widths, and do not choose permutations using test labels. A local proxy does not verify production scaling laws.
Illustrative example: Illustration: Swapping price and age while keeping the original lookup can interpret age 18 as a price.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does permutation grouping change interaction opportunities versus fully shared RankUp input?
RankUp: Towards High-rank Representations for Large Scale Advertising Recommender Systems
Show answer and example
Answer: New partitions let previously separated fields meet, independent embeddings add views, and global tokens connect groups. Shared inputs can favor strong fields, while groups can miss or duplicate crosses. Combined local changes do not attribute gains to one component.
Illustrative example: Illustration: Another permutation can put age with category; unaligned parameter shuffling can instead destroy meaning.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why isn't higher RankUp representation rank proof of useful interests?
RankUp: Towards High-rank Representations for Large Scale Advertising Recommender Systems
Show answer and example
Answer: Rank measures independent directions, not task information. Noise can raise rank and obscure useful structure. Inspect spectra with task and subgroup metrics rather than claiming quality when tokens become less alike.
Illustrative example: Illustration: Random orthogonal user noise can yield full rank without adding any predictive preference.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why separate representation compression from recommendation adaptation in MARC, and why might an intermediate LLM layer be chosen?
Modular Representation Compression: Adapting LLMs for Efficient and Effective Recommendations
Show answer and example
Answer: The last layer favors the language objective, whereas an intermediate layer can retain more compressible general features. What compression preserves is not necessarily what recommendation needs. Separate modules and information constraints distinguish dimensional cost from task relevance. Intermediate-layer superiority must be measured per model and dataset.
Illustrative example: Illustration: The final layer excels at next-word prediction, but a middle layer may retain product use. Reconstruction alone can preserve costly language details irrelevant to purchase preferences.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
How should MARC avoid mixing validation and test during layer and nuisance-projection selection?
Modular Representation Compression: Adapting LLMs for Efficient and Effective Recommendations
Show answer and example
Answer: Estimate directions from training data and choose layer, dimension, and removed directions on validation, then freeze for test. Match projections to the selected hidden-state space. A fixed-score proxy without full information-objective training must be described within that boundary.
Illustrative example: Illustration: Choosing the highest test NDCG layer turns test into tuning. Pick it first, then check transfer on independent data.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
What are the benefits and risks of separate MARC compression and adaptation versus one linear reducer?
Modular Representation Compression: Adapting LLMs for Efficient and Effective Recommendations
Show answer and example
Answer: One projection conflates information loss with task learning. Separate modules make each effect measurable, but add choices and objective interactions. Removing a supposed nuisance can delete useful signals. The public proxy was negative, so information constraints alone do not establish generalization.
Illustrative example: Illustration: Removing content-specific style may help matching unless style itself drives user preference. Ablate both stages.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why might MARC choose an intermediate layer instead of assuming the LLM's final layer is best?
Modular Representation Compression: Adapting LLMs for Efficient and Effective Recommendations
Show answer and example
Answer: Final layers specialize in pretraining outputs. MRA observations motivate separating compression and adaptation to remove proxy-task interference while retaining recommendation information. Select layers on validation; middle layers are not universally superior.
Illustrative example: Illustration: A middle layer may distinguish two user groups better in a compact representation even if the final layer is best at predicting words.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Core mechanism
Why doesn't label-free FUSE get more reliable simply by adding correlated verifiers?
FUSE: Ensembling Verifiers with Zero Labeled Data
Show answer and example
Answer: Moment estimation needs identifiability assumptions such as a better-than-random majority and conditionally independent triplets. Transforms mitigate correlation, but shared errors can remain. Posteriors are not gold, which stays evaluation-only.
Illustrative example: Three verifiers copying one wrong rule are not three independent correctness signals.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Implementation details
Can gold choose score transforms or ensemble parameters?
FUSE: Ensembling Verifiers with Zero Labeled Data
Show answer and example
Answer: Not in the documented label-free fit. Use score matrices only and reserve labels for independent evaluation. Choosing transforms by gold on the same questions breaks both boundaries.
Illustrative example: Trying transforms and selecting the best by answers is not unsupervised selection.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Method comparison
Do moment-based posteriors provide the same guarantees as gold labels?
FUSE: Ensembling Verifiers with Zero Labeled Data
Show answer and example
Answer: No. They depend on identifiability; high correlation or constant scores can break estimation. High estimated probability is not truth or safety.
Illustrative example: Shared wrong rules can produce agreement on the wrong answer.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Basics
Why not average verifier scores in FUSE?
FUSE: Ensembling Verifiers with Zero Labeled Data
Show answer and example
Answer: Correlated errors get counted repeatedly. Estimate query-conditioned quality from scores, transform to mitigate dependencies, then build posteriors for ensemble optimization.
Illustrative example: Three near-identical verifiers need not add three times the evidence.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
Why does StepPO aggregate token ratios within a step before clipping the step ratio?
Show answer and example
Answer: The environment executes a whole action, so independent token clipping differs from action-level probability change. Average within-step log ratios and exponentiate, then compute value and GAE at step boundaries, aligning credit with environment transitions. Exclude tool observations from action tokens and preserve consistent length normalization.
Illustrative example: A shell command has many tokens but executes once when complete. Evaluate the step's overall change rather than assigning its state reward independently to each subword.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
How does StepPO keep observation tokens out of action-ratio aggregation?
Show answer and example
Answer: Mark step and response-token boundaries and aggregate only policy ratios of tokens in the action. Observations are conditions, not generated actions. Deterministic local tool plans verify a step state machine but not real LLM token rollouts.
Illustrative example: A thousand-character tool response must not become a thousand policy tokens in the action's ratio mean.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
Is clipping token ratios before averaging equivalent to aggregating and clipping a step ratio?
Show answer and example
Answer: No. Clipping is nonlinear, so order changes which actions are constrained. Average log ratios and exponentiate to obtain a geometric step ratio, not the arithmetic mean of raw ratios.
Illustrative example: For one large and one small ratio in an action, geometric aggregation can produce a different update from individual clipping.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why does StepPO use observation–action as the natural decision unit?
Show answer and example
Answer: Tool actions span tokens while environment feedback follows the complete action. StepPO computes value and GAE at step boundaries and aggregates action-token ratios into a step ratio before clipping.
Illustrative example: A tool call has a name and arguments, but the environment returns feedback only after the complete invocation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Core mechanism
Why not recompute GroupDPO coefficients inside each response microbatch?
GroupDPO: Memory-Efficient Group-Wise Direct Preference Optimization
Show answer and example
Answer: Freeze derivatives from the entire group at the same parameter point and replay response gradients to recover the first-order result. Local recomputation changes sets and normalization. Recompute after every update. Loss values and Hessians are not equivalent.
Illustrative example: Each response carries fixed full-group credit rather than re-ranking within a small compute batch.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Implementation details
Can GroupDPO coefficients be reused after the optimizer step?
GroupDPO: Memory-Efficient Group-Wise Direct Preference Optimization
Show answer and example
Answer: No. Derivatives change with parameters. Recompute. Do not divide microbatch backpropagation by local groups or microbatch count; positive NLL also uses the full-batch group normalization.
Illustrative example: Reusing old coefficients saves a pass but becomes a stale approximation, not exact current gradients.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Method comparison
Are GroupDPO Margin, MPO, Softmax and All-Pairs equivalent objectives?
GroupDPO: Memory-Efficient Group-Wise Direct Preference Optimization
Show answer and example
Answer: No. They compare sets differently. What is shared is precomputing derivatives of the chosen full-group objective and replaying separately. Changing the objective differs from changing compute batches while retaining its first derivative.
Illustrative example: All pairwise comparisons may assign different coefficients than pooling positives first; the same memory method serves both.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Basics
Does GroupDPO concatenate all responses into one long response?
GroupDPO: Memory-Efficient Group-Wise Direct Preference Optimization
Show answer and example
Answer: No. One prompt has nonempty positive and negative sets; responses retain separate probabilities and IDs. The joint objective uses set relationships without requiring simultaneous activation storage.
Illustrative example: Two positive and two negative answers are encoded independently while the objective compares both sets.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Core mechanism
Why not use exact match against reference text for speech-to-speech translation?
Show answer and example
Answer: Audio output needs real ASR before scoring with prescribed semantic translation metrics. Both transcription and translation errors matter. Exact text matching rejects valid paraphrases and bypasses audio. Few-shot examples come from development data, not test references.
Illustrative example: Two spoken versions of “Please come tomorrow” can both be correct with different wording; preserve both the semantic and audio pipeline.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Implementation details
What limits apply to NaijaS2ST five-shot examples and proxy language tokens?
Show answer and example
Answer: Take five examples from DEV, never formal test. Some Seamless adaptations map Hausa to Arabic and Naijá to French tokens as conventions, not native model support. Local work has not downloaded audio or run complete training and evaluation.
Illustrative example: Assigning an Arabic control token for Hausa fine-tuning does not mean the audio is translated to Arabic or that untrained Hausa support exists.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Method comparison
Do SSA-COMET and ChrF++ measure the same kind of agreement?
Show answer and example
Answer: SSA-COMET uses a multilingual semantic model; ChrF++ mainly uses character and word n-gram overlap. The scores are complementary and not interchangeable accuracy percentages. Speech output still needs the prescribed ASR path.
Illustrative example: Two valid paraphrases can preserve meaning with different character overlap; the larger number is not automatically a larger fraction correct.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Basics
What does accent diversity require in NaijaS2ST evaluation?
Show answer and example
Answer: Pronunciation variation affects understanding and speech evaluation. Cover multiple speakers and accents, report translation directions separately, and distinguish speech-to-text from speech-to-speech. One direction or a few speakers cannot characterize all Nigerian languages or accents.
Illustrative example: English→Hausa is not Hausa→English, and understanding one accent does not prove reliable synthesis of another.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why does LWGR learn personalized soft instructions while limiting degradation against a frozen reference?
LWGR: Lagrangian-Constrained Personalized World Knowledge for Generative Recommendation
Show answer and example
Answer: A fixed prompt cannot adapt across users. Subspace codebooks produce trainable soft tokens to personalize knowledge extraction. LLM knowledge can conflict with behavioral evidence, so the frozen reference anchors a confidence degradation constraint; nearline computation controls serving cost. Learned prompts do not make extracted knowledge true or more reliable.
Illustrative example: Illustration: Two travelers care about price and photography respectively. Soft prompts can retrieve different knowledge, while the constraint should discourage new evidence from overriding reliable existing scores.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
How should LWGR's straight-through code selection and dual update be checked separately?
LWGR: Lagrangian-Constrained Personalized World Knowledge for Generative Recommendation
Show answer and example
Answer: Use hard codes forward and soft-distribution gradients backward; verify soft-instruction parameters change. Update λ using C−ε and project to nonnegative values. A rising multiplier signals violation, not satisfaction. Frozen reference, BOS fusion, and nearline cache are separate interfaces.
Illustrative example: Illustration: If C remains above tolerance while λ rises, report failure to satisfy the constraint. A hard index cutting gradients also prevents learning even though it selects codes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
What does LWGR's reference confidence constraint protect compared with ordinary KL?
LWGR: Lagrangian-Constrained Personalized World Knowledge for Generative Recommendation
Show answer and example
Answer: It penalizes the amount by which a positive target's mean token log-probability falls below the reference tolerance. KL instead measures whole-distribution drift. Updating the multiplier is not sufficient: locally λ changed without an improvement in degradation, so protection was not demonstrated.
Illustrative example: Illustration: Moving probability to another reasonable candidate may raise KL without much target damage; a large target drop directly activates the one-sided constraint.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
How can LWGR's soft instruction express user differences beyond handwritten wording?
LWGR: Lagrangian-Constrained Personalized World Knowledge for Generative Recommendation
Show answer and example
Answer: Split user context into subspaces and select codewords from independent codebooks. The resulting differentiable prompt tokens and history generate user knowledge; recommendation supervision can change them. Capacity and training coverage limit personalization, and the tokens need not correspond to readable instructions.
Illustrative example: Illustration: A photography novice and a professional may select different soft tokens even though the handwritten prompt says 'analyze interests' for both.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why does GenRec target a next-page sequence and keep NLL alongside GRPO?
GenRec: A Preference-Oriented Generative Framework for Large-Scale Recommendation
Show answer and example
Answer: Item targets alone do not teach order and complementarity within pages. Sequence targets model composition, group-relative reward shapes full-page utility, and NLL anchors to logged behavior against reward drift. Rewards that omit relevance or feasibility can still mislead.
Illustrative example: A page objective can replace one of three redundant items; an NLL anchor can restrain a novelty-only reward from recommending irrelevant content.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Why compare GenRec page advantages only within the same context?
GenRec: A Preference-Oriented Generative Framework for Large-Scale Recommendation
Show answer and example
Answer: Same-context pages share history and candidate difficulty, so mean subtraction measures relative quality. Mixing users confounds request difficulty. Update prefix and duplicate masks together, apply NLL to legal targets, and do not use Token Merger to expose future test pages.
Illustrative example: Compare pages within each photography or cold-start user rather than marking every harder request negative.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
What drift does GenRec's NLL limit compared with optimizing GRPO-SR reward alone?
GenRec: A Preference-Oriented Generative Framework for Large-Scale Recommendation
Show answer and example
Answer: Group-relative rewards compare page utility while NLL preserves logged behavior, limiting drift toward sparse or biased rewards. Their balance trades imitation for exploration. Local relative gains depend on proxy labels and baseline and do not establish real satisfaction.
Illustrative example: Diversity reward alone can produce irrelevant novelty; NLL retains basic interests.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Why organize consecutive positives as a next-page target in GenRec?
GenRec: A Preference-Oriented Generative Framework for Large-Scale Recommendation
Show answer and example
Answer: Within-page consumption has order and complementarity that point prediction misses. Generate with prefix dependence and score the entire page with grouped rewards. Pages assembled from consecutive public actions are proxies, not observed impression pages.
Illustrative example: Tent, sleeping bag, and pad can form a useful page; three consecutive movie views do not prove simultaneous display.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why generate attributes before SID and constrain quantization capacity?
Show answer and example
Answer: Category, seller, and brand condition generation explicitly. Capacity prevents popular codes monopolizing the space; task context handles differing needs. Early attribute errors propagate, so interpretable steps do not guarantee correct items.
Illustrative example: Choosing dress shoes instead of sneakers early can derail the path despite precise later decoding.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Can future target behavior reveal attributes used as inputs?
Show answer and example
Answer: No. Attributes come from allowed item metadata. Target attributes can supervise generation, not be given as known answers. Public MovieLens lacks private multi-scenario serving.
Illustrative example: Supplying the true next item's brand leaks a clue.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
How does capacity-limited quantization differ from maximizing popular-code likelihood?
Show answer and example
Answer: Capacity limits reduce hotspots and disperse usage, while likelihood alone follows frequency. Evaluate relevance and balance together.
Illustrative example: Avoid crowding a bucket without randomly distributing items just to equalize counts.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Why generate an attribute chain before SIDs?
Show answer and example
Answer: Explicit category, seller, and brand context can bridge generative and discriminative representations. The chain guides content and does not establish independent language reasoning.
Illustrative example: Generate category and brand before the detailed semantic path.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Core mechanism
Why does HILL train index nodes with users and interaction towers instead of only clustering item vectors?
Efficient Retrieval Scaling with Hierarchical Indexing for Large Scale Recommendation
Show answer and example
Answer: Pure item geometry may not match an interaction model's choices. Train node proxies in user–item scoring and let deeper residuals recover information missed above. Beam search can still omit paths; isolated test answers cannot guide any test-time updates or selection.
Illustrative example: Similar products may serve different users, so interaction-informed nodes can organize them better than content proximity alone.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Implementation details
Why validate HILL soft assignments and hierarchical beam separately?
Efficient Retrieval Scaling with Hierarchical Indexing for Large Scale Recommendation
Show answer and example
Answer: Continuous training assignments create pseudo-items and gradients; serving explores a limited discrete tree. Check normalization, residual updates, parent-child mapping, and actual budgets independently, including small-catalog variance. Good reconstruction does not establish beam coverage.
Illustrative example: A target can receive small mass from several soft nodes and be absent from every retained hard branch.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Method comparison
Why can HILL residual layers complement each other instead of repeating raw-vector clustering?
Efficient Retrieval Scaling with Hierarchical Indexing for Large Scale Recommendation
Show answer and example
Answer: Each layer quantizes the prior layer's unexplained residual, while repeated clustering can relearn dominant directions. Complementarity is not guaranteed and beam budgets can miss items. Local compression worked but quality regressed; fewer candidates are not automatically better.
Illustrative example: After a coarse electronics node, residuals can express use and price rather than repeatedly learning the broad theme.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Basics
Why involve user and interaction signals in HILL index-node training?
Efficient Retrieval Scaling with Hierarchical Indexing for Large Scale Recommendation
Show answer and example
Answer: ANN organizes item neighbors geometrically, which may disagree with complex relevance. Scoring pseudo-items with users transfers retrieval relations to the tree before narrowing candidates. Training, tree versions, and serving need to stay aligned.
Illustrative example: Two content neighbors with different audiences can be organized by recommendation value, not just item distance.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Core mechanism
Why isn't cached teacher scoring equivalent to continuously online distillation?
Show answer and example
Answer: Initial student samples are scored under one teacher config. Later updates shift visitation without refreshing labels. Coverage and teacher consistency govern the cost tradeoff; local evidence is data flow only.
Illustrative example: Cached early error prefixes help initially but later new approaches need an explicit refresh protocol.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Implementation details
What Lightning cache identities must match?
Show answer and example
Answer: Teacher checkpoint, prompt, tokenizer, response and position mask. Changes invalidate labels. Local data flow is not Qwen vocabulary distillation or AIME.
Illustrative example: A tokenizer update can misalign stored token scores despite the same filename.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Method comparison
How does offline cache coverage differ from refreshing rollouts each step?
Show answer and example
Answer: Cache covers initial student visits; updated policy may reach new states. Refreshing covers those at higher cost. Cached isn't always on-policy.
Illustrative example: A newly learned solution's prefixes won't appear automatically in an old cache.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Basics
Why cache Lightning OPD teacher scores before updates?
Show answer and example
Answer: Online teacher inference can bottleneck training. Score initial student traces once and reuse labels. This front-loads cost rather than eliminating generation/scoring.
Illustrative example: Prepare teacher scores per prefix and reread them for multiple updates.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Core mechanism
Why cache predicted pair latents instead of one user vector?
SOLARIS: Speculative Offloading of Latent-bAsed Representation for Inference Scaling
Show answer and example
Answer: Deep interactions depend on a particular item. Predict likely pairs, compute asynchronously, use hits and fallback on misses. Coverage, freshness, and budget matter. Future outcomes can supervise training, not online selection.
Illustrative example: Precompute ten likely products, but serve an unexpected one through fallback.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Implementation details
What should a SOLARIS cache key capture?
SOLARIS: Speculative Offloading of Latent-bAsed Representation for Inference Scaling
Show answer and example
Answer: User/item identity, model and feature versions, and validity; define hit and fallback paths. An abstract implementation or hit rate does not prove service latency.
Illustrative example: Do not silently reuse a latent from a previous model version.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Method comparison
How does a latent cache differ from a final list cache?
SOLARIS: Speculative Offloading of Latent-bAsed Representation for Inference Scaling
Show answer and example
Answer: A latent is a middle feature the lightweight ranker can combine with current inputs. A cached list fixes ordering. Latents still age and need version consistency.
Illustrative example: Reusing a latent with today's price can change yesterday's order.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Basics
Why predict future user–item pairs before computing SOLARIS latents?
SOLARIS: Speculative Offloading of Latent-bAsed Representation for Inference Scaling
Show answer and example
Answer: Computing all pairs online is expensive. Predict likely pairs and asynchronously precompute them for lightweight consumption. Misses need fallback and wrong guesses waste budget.
Illustrative example: Prepare representations for a shortlist instead of an entire catalog.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why make HID/SID fusion target-dependent?
SID-Coord: Coordinating Semantic IDs for ID-based Ranking in Short-Video Search
Show answer and example
Answer: Head items have rich behavior, while tail items need semantic sharing. Target-aware gating, hierarchical fusion, and history alignment coordinate both without guaranteeing all tail gains.
Illustrative example: One user can receive more ID memorization for a hit film and more semantic weight for a new film.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
How align SID level weights and sample gates?
SID-Coord: Coordinating Semantic IDs for ID-based Ranking in Short-Video Search
Show answer and example
Answer: Level weights correspond to one item's SID levels; the gate belongs to its user–target pair, not a batch constant. MovieLens proxies do not reproduce production codes or traffic.
Illustrative example: Giving A's fine code to B misaligns semantics despite matching shapes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
Are hierarchical α and HID–SID gate the same axis?
SID-Coord: Coordinating Semantic IDs for ID-based Ranking in Short-Video Search
Show answer and example
Answer: No. α mixes SID levels within an item, while the gate balances ID memory and semantics per sample. A single constant loses both distinctions.
Illustrative example: Mix coarse/fine semantics before mixing with the hash-ID branch.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why keep HID and SID together?
SID-Coord: Coordinating Semantic IDs for ID-based Ranking in Short-Video Search
Show answer and example
Answer: HID memorizes head interactions while hierarchical SID shares semantics for the tail. Target gating and history alignment balance them per example, not by replacing IDs.
Illustrative example: Popular items use memorization; new ones rely more on semantic neighbors.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
Why update both graph memory and policy with new rollouts rather than freeze the tool graph?
Show answer and example
Answer: Executions reveal usable and failed tool transitions, which should change edge weights. The policy also learns to use updated relationships. Fixed graphs limit exploration to older experience, while updating the graph alone does not ensure consumption. Both sides need real rewards, not preset successful edges.
Illustrative example: If a tool's output can newly feed another tool, adding the edge is not enough; the policy should learn to select that route.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
How can SEARL show later decisions actually consume graph updates rather than only logging writes?
Show answer and example
Answer: Compare edge weights before and after rollouts, later retrieval hits, and policy inputs to confirm that updated experience is available. Nonzero telemetry proves events, not real browsing or multi-GPU RL capability from the mini-suite.
Illustrative example: If edge weights change on disk but the policy reads stale cache, the experience loop did not close.
Sections in the Chinese source: 核心公式, 复现边界. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
How does SEARL's joint graph and policy update differ from editing memory only?
Show answer and example
Answer: Graph changes alter retrieved external information. Joint updates also change how the policy uses it and acts. Training the policy with a fixed graph likewise misses experience structure evolution. Record both state histories separately.
Illustrative example: A new successful edge does not show that policy parameters learned to use it in relevant states.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why does SEARL store tool experience in a graph with successful transitions?
Show answer and example
Answer: The graph represents relationships between tools and effective transitions for retrieval and policy decisions. New rollouts update edge weights and the policy, so experience changes with actual outcomes rather than remaining static.
Illustrative example: Repeated success for one tool sequence can make that relationship easier to retrieve for later tasks.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Core mechanism
Why can't a committed action be executed again unconditionally after a crash?
LogAct: Enabling Agentic Reliability via Shared Logs
Show answer and example
Answer: Commit records approval but may not reveal whether the external side effect happened. A call can succeed before its receipt is logged. Recovery needs idempotency keys or transaction contracts to distinguish those cases. Votes govern approval, not external exactly-once execution.
Illustrative example: If payment succeeds and the process crashes before logging a result, blind replay may charge twice. Use idempotency or transaction lookup.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Implementation details
How does local LogAct handle quorum and crash recovery?
LogAct: Enabling Agentic Reliability via Shared Logs
Show answer and example
Answer: Require k approvals from N named voters, deduplicate voters, and disallow vote changes. Started without Result does not retry by default. Only explicit recovery for a known idempotent action, after excluding live executors, uses a stable key. Arbitrary dynamic policy languages are unsupported.
Illustrative example: One person's two votes do not count twice, and a commit does not justify replaying unknown side effects.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Method comparison
How does LogAct k-of-N approval differ from waiting for every vote?
LogAct: Enabling Agentic Reliability via Shared Logs
Show answer and example
Answer: A fixed threshold permits commit as soon as k approvals arrive. Reject early if approvals plus remaining votes cannot reach k. Irrelevant pending votes need not finish, and a voter counts once. This does not implement all dynamic approval policies.
Illustrative example: With three voters and a threshold of two, two approvals suffice. Two rejections make the last vote unable to reach two approvals.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Basics
Why does LogAct separate vote commit from execution?
LogAct: Enabling Agentic Reliability via Shared Logs
Show answer and example
Answer: Persist intent, review, and commit before execution, separating authorization from external effects and enabling crash audit. Agreement in conversation is not a trusted durable commit.
Illustrative example: Record approval before an executor calls a payment API.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Core mechanism
How does Dual-Rerank's parallel NAR student absorb an autoregressive teacher's ordering dependencies?
Dual-Rerank: Fusing Sequential Dependencies and Utility for Generative Reranking
Show answer and example
Answer: The AR teacher models prefix-dependent item interactions at sequential serving cost. Ranking distribution and utility distillation compress its ordering preferences into parallel rank estimates. The student approximates rather than recomputes every prefix-dependent decision.
Illustrative example: Learn that a tutorial before equipment raises list value and emit the list once; novel combinations may still differ from the teacher.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Why align candidates, positions, and masks for Dual-Rerank's AR–NAR KL?
Dual-Rerank: Fusing Sequential Dependencies and Utility for Generative Reranking
Show answer and example
Answer: The teacher is prefix-constrained and the student may see all candidates in parallel. Compare the same event space with matching masks for padding and selected items and matching candidate identities. Copying final teacher scores to each position does not teach order.
Illustrative example: A step-two teacher excludes the movie at step one; a full-pool student distribution has a different denominator until explicitly aligned.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
What trade-off does utility distillation allow beyond matching AR probabilities?
Dual-Rerank: Fusing Sequential Dependencies and Utility for Generative Reranking
Show answer and example
Answer: KL matches distributions, while utility lets quality and serving constraints enter the student's objective so it can deviate where appropriate. Bad utility proxies can harm quality, and a latency term is not proof of actual speed. Measure both contributions separately.
Illustrative example: The student may preserve high-value list relationships without matching tail probabilities exactly; speed alone can reward irrelevant output.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why does Dual-Rerank pair an AR teacher with an NAR student?
Dual-Rerank: Fusing Sequential Dependencies and Utility for Generative Reranking
Show answer and example
Answer: The teacher learns prefix and order dependencies item by item. The student estimates positions in parallel using distilled distributions and utilities, avoiding teacher calls at serving. Check deduplication and position consistency rather than assuming full teacher capability is preserved.
Illustrative example: The student can learn basic-then-advanced tutorial ordering and score positions together.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Core mechanism
Why must a summary forecast and pass independently after a long reasoning trace meets reference gates?
TFRBench: A Reasoning Benchmark for Evaluating Forecasting Systems
Show answer and example
Answer: Summaries can omit decisive evidence and cannot inherit original correctness. Numerical accuracy and reasoning consistency jointly constrain references. Label-informed filtering measures construction quality, not held-out ability. Reject zero historical scales rather than adding arbitrary epsilon.
Illustrative example: A long trace explains seasonality and predicts correctly, while a summary saying only 'upward trend' may predict badly. Check compression independently.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Implementation details
Can TFRBench add epsilon and award a high score when historical seasonal scale is zero?
TFRBench: A Reasoning Benchmark for Evaluating Forecasting Systems
Show answer and example
Answer: No. MASE is undefined and the sample must be rejected. Reference admission strictly requires MASE below one and S at least four. Document composite ranking weights; local settings are not the authors' unique formula. Future truth remains with the grader.
Illustrative example: Constant histories have no difference scale. Inventing a denominator does not make their errors comparable.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Method comparison
How does TFRBench MASE differ in scale from comparing raw MAE across series?
TFRBench: A Reasoning Benchmark for Evaluating Forecasting Systems
Show answer and example
Answer: MASE divides error by historical mean absolute seasonal differences, defining thresholds relative to each series. MAE retains original units and is not directly comparable across dimensions. MASE below one is a numeric gate alongside separate consistency. Zero historical scale makes it undefined.
Illustrative example: An error of ten has different meaning for volatile sales and nearly constant temperature. Normalize by each series' historical seasonal changes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Basics
Why does TFRBench check both numerical error and reasoning consistency?
TFRBench: A Reasoning Benchmark for Evaluating Forecasting Systems
Show answer and example
Answer: A close forecast may be luck and a fluent explanation may forecast badly. References need both criteria. Labels and feedback used in construction do not yield held-out ability scores.
Illustrative example: A lucky trend guess without evidence and a justified forecast with large errors both fail full admission.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Core mechanism
Why can't citation checks and revision acceptance replace real experiments and research arguments?
PaperOrchestra: A Multi-Agent Framework for Automated AI Research Paper Writing
Show answer and example
Answer: Citation identity, abstracts, and cutoff checks verify sources and time eligibility. Revision scores constrain edits to existing content. Five writing roles organize actual ideas and logs; they do not create results. Mechanical BibTeX checks and judges are not full literature synthesis or research capability.
Illustrative example: Example: A chart agent can plot real logs, but cannot invent metrics to complete a manuscript. Authentic citations do not make fabricated results true.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Implementation details
What conditions govern citation verification and revision rollback, and what was completed locally?
PaperOrchestra: A Multi-Agent Framework for Automated AI Research Paper Writing
Show answer and example
Answer: Citations must match real entities, include abstracts, predate the cutoff strictly, and deduplicate by paper ID. Revert revisions when scores fall or ties regress across subdimensions, at most three rounds. Local work reviews literature and costs without five agents, writing models, or metrics. A BibTeX checker is not literature synthesis.
Illustrative example: Example: Deduplicate two URLs for one paper and exclude post-cutoff research. Valid formatting does not mean the citation supports a sentence.
Sections in the Chinese source: 核心公式, 本地复现. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Method comparison
Why aren't automated judging and human preferences equivalent evidence of writing quality?
PaperOrchestra: A Multi-Agent Framework for Automated AI Research Paper Writing
Show answer and example
Answer: Automated judges can favor form and fluency, while people assess literature reviews differently. The paper reports weak correlation. Higher revision score still needs factual and dimension-specific checks. One judge improvement does not verify real paper quality or scientific correctness.
Illustrative example: Example: More topical vocabulary may boost automatic scores while misaligned citations and overstated results make experts rate a review worse.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Basics
Why does PaperOrchestra organize actual experiment material into an argument instead of just writing fluent prose?
PaperOrchestra: A Multi-Agent Framework for Automated AI Research Paper Writing
Show answer and example
Answer: Scientific writing connects question, method, evidence, and citations. Fluent text can conceal missing tests or unsupported conclusions. PaperOrchestra coordinates roles to write, verify, and revise from ideas and logs. Automation cannot supply absent validation, and a citation may not support the claim.
Illustrative example: Example: With logs for two datasets, explain those two. Stronger wording cannot create a third experiment or generalization result.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why does MBGR find the nearest future event for each business rather than just supervising the next log event?
MBGR: Multi-Business Generative Recommendation
Show answer and example
Answer: Interleaved streams let frequent businesses dominate adjacent labels. LDR finds each business's nearest future target from a shared current state and conditions on business identity, exposing multiple prediction paths. Mask missing futures. This is training supervision only; future events are not current inputs.
Illustrative example: Illustration: A video may always be next, while a product search comes later. Per-business targets preserve shopping supervision without feeding future purchase IDs into the current recommender.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Why not fill a missing future business event with padding or the last item in MBGR?
MBGR: Multi-Business Generative Recommendation
Show answer and example
Answer: There is no observed target for that business in the window. Filling it creates a false transition, so mask its loss. t′ must be the earliest position strictly after the current time with the right business identity, not merely a similar category. Training label availability differs from inference input visibility.
Illustrative example: Illustration: If no purchase follows, the last movie is not a shopping positive. Masking preserves 'unobserved' rather than inventing an answer.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
What does business-conditioned MoE share compared with training a separate generator for each business?
MBGR: Multi-Business Generative Recommendation
Show answer and example
Answer: A shared expert pool learns common patterns, while business-conditioned gates produce specific representations and a conditioned autoencoder preserves SID semantics. Separate models duplicate learning but reduce conflict; shared models risk negative transfer. Genres proxy for businesses, not real cross-product labels. The local lack of gain should be retained.
Illustrative example: Illustration: Shopping and content may reuse one semantic expert at different weights. Separate models cannot exchange knowledge, but too much sharing can let high-traffic content suppress shopping.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Why look up the nearest future interaction separately for each business in MBGR?
MBGR: Multi-Business Generative Recommendation
Show answer and example
Answer: A global next-item label covers only one business. LDR looks ahead per business so a shared state learns multiple directions and masks missing labels. Multi-task supervision does not let future events enter history.
Illustrative example: Illustration: A current state can learn about both the next short video and later shopping instead of letting abundant video labels drown out shopping.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
Why aren't correctness and incorrectness critiques just repeated versions of the same decision?
Reasoning-Driven Synthetic Data Generation and Evaluation
Show answer and example
Answer: A single critic may accept surface plausibility. Separately asking for evidence of correctness and looking for errors gives complementary filters. Admission requires satisfying requirements, positive correctness, and negative incorrectness judgments. Factor trees ensure coverage and batch-relative difficulty calibrates complexity; rule checks cannot replace semantic critiques as quality evidence.
Illustrative example: A polished question may contain mutually exclusive requirements. An error-seeking critic should uncover them rather than echoing 'looks good.'
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
If Simula's two critics share a base model, may the second see the first verdict?
Reasoning-Driven Synthetic Data Generation and Evaluation
Show answer and example
Answer: No. Call independently with only the sample. Admission requires the requirements critic to pass, correct to affirm, and incorrect to deny. Semantic generation and critique are central; fixture callbacks or zero 13-gram overlap on short text do not establish full quality or absence of contamination.
Illustrative example: Telling the error critic that the other critic already passed the sample breaks independence.
Sections in the Chinese source: 核心公式, 数据协议与调用边界. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
How does Simula's Elo-based relative complexity differ from absolute model-assigned difficulty?
Reasoning-Driven Synthetic Data Generation and Evaluation
Show answer and example
Answer: Elo updates within-batch rankings through pairwise outcomes and depends on matchups and settings. Absolute scores use a different scale. Local initial 1000, scale 400, and k=32 are explicit settings, not claimed as the authors' unique values.
Illustrative example: Harder than other examples in this batch does not give a universal unit of difficulty across tasks.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why does Simula build a factor taxonomy before generating examples?
Reasoning-Driven Synthetic Data Generation and Evaluation
Show answer and example
Answer: Direct seed expansion can limit coverage and produce similar samples. A tree organizes factors and combinations; generation controls global node combinations and local scenarios before independent critique and relative complexity calibration.
Illustrative example: Separate task type, conditions, and difficulty factors before combining them into distinct cases instead of repeatedly rewording one item.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why put reversed conversion-related history into RCLRec's decoder prefix and weight curriculum quality?
RCLRec: Reverse Curriculum Learning for Modeling Sparse Conversions in Generative Recommendation
Show answer and example
Answer: Sparse conversion labels weakly supervise long histories. Selecting a related short subsequence and reversing it foregrounds events closer to conversion and creates intermediate supervision. Quality weighting limits weak curricula, but mistaken relevance can reinforce bias.
Illustrative example: Illustration: Start from cart addition and work back through comparison and search, while downweighting accidental browsing.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Why preserve event timing and visibility when implementing reverse sequences in RCLRec?
RCLRec: Reverse Curriculum Learning for Modeling Sparse Conversions in Generative Recommendation
Show answer and example
Answer: Reversal changes prefix order, not what was observed before the target. Define selection, order, quality masks, and causal decoding for the transformed layout. Reversing a tensor with mismatched positions is not the specified curriculum.
Illustrative example: Illustration: Pre-purchase comparisons can be reversed, but a return after purchase cannot join the prefix.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
How does quality weighting reduce uninformative curriculum supervision compared with equal weights?
RCLRec: Reverse Curriculum Learning for Modeling Sparse Conversions in Generative Recommendation
Show answer and example
Answer: q(C) limits irrelevant segments' influence on the main objective. It can be mistaken or overly concentrated and reduce coverage. Preserve the negative public proxy rather than assuming effectiveness.
Illustrative example: Illustration: Ten unrelated clicks do not deserve more auxiliary weight than one clear compare-to-cart chain.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Does zero RCLRec curriculum quality stop the main objective from learning?
RCLRec: Reverse Curriculum Learning for Modeling Sparse Conversions in Generative Recommendation
Show answer and example
Answer: No. Main loss plus λq(C) times the curriculum loss retains the main term at q(C)=0. Do not mask the whole sample. For gradient claims, distinguish whether the weight is trainable.
Illustrative example: Illustration: A poor curriculum can be ignored while the true purchase label still trains the primary objective.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why can cross-domain distillation omit source training data but not blindly trust the source teacher?
Zero-shot Cross-domain Knowledge Distillation: A Case study on YouTube Music
Show answer and example
Answer: The teacher can provide task and auxiliary signals directly on target examples, so raw source logs are unnecessary. The teacher may still be miscalibrated or lack important new-domain features. Use target supervision, and do not hide negative local comparisons behind teacher size.
Illustrative example: Video-domain preferences can inform music, but equating video viewing habits with music listening can transfer errors.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
Why does temperature distillation use `T²`, and how should auxiliary logits align in cross-domain KD?
Zero-shot Cross-domain Knowledge Distillation: A Case study on YouTube Music
Show answer and example
Answer: Temperature changes softmax sharpness and gradient scale; `T²` compensates for the latter. Teacher and student tasks and classes must correspond semantically, not merely by shape. Serving only needs the student.
Illustrative example: A teacher long-watch head should not be matched to a student payment head. Higher temperature cannot fix a task mismatch.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
What do target-only student and same-domain DIN controls test in cross-domain KD?
Zero-shot Cross-domain Knowledge Distillation: A Case study on YouTube Music
Show answer and example
Answer: The target-only comparison isolates added distillation knowledge; DIN anchors overall model quality. Internal gains do not mean matching a stronger baseline, and the source reports very different results under the two controls. A temperature ablation cannot replace matched data, candidates, and budget.
Illustrative example: A student improving from 10 to 11 is an internal gain, but does not beat a DIN score of 30.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why can cross-domain KD run a source teacher on target examples without source logs?
Zero-shot Cross-domain Knowledge Distillation: A Case study on YouTube Music
Show answer and example
Answer: Teacher output distributions and auxiliary signals carry knowledge without replaying source training data. Target labels constrain the real serving task and the teacher must understand available target features. Removing data access does not verify new-domain calibration.
Illustrative example: A video teacher provides soft preferences to a music student without sharing video logs; some video preferences may not transfer.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Core mechanism
Does Switch Attention choose one path per layer or per token?
Switch Attention: Towards Dynamic and Fine-grained Hybrid Transformers
Show answer and example
Answer: A fine-grained router chooses full or local for each token. Training must account for discrete selection; actual savings depend on global call rate and overhead, and sparsity does not guarantee quality.
Illustrative example: Nearby conjunctions can use local reads, while citing an early name may need global history in the same layer.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Implementation details
Can the local soft router demonstrate a branch-selective kernel speedup?
Switch Attention: Towards Dynamic and Fine-grained Hybrid Transformers
Show answer and example
Answer: No. It checks mixing without hard routing, 32K continued pretraining, or fused selective kernels. Verify actual executed branches and runtime separately from gate values.
Illustrative example: A 90% local weight does not prove 90% of global calls were skipped.
Sections in the Chinese source: 复现边界. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Method comparison
How does hard switching differ from continuously mixing both branches?
Switch Attention: Towards Dynamic and Fine-grained Hybrid Transformers
Show answer and example
Answer: Hard selection can execute only one branch. Continuous mixing generally computes both and weights outputs. Gradients and savings differ; the local soft router is not the paper's STE hard router.
Illustrative example: Multiplying an already computed attention output by nearly zero does not save its execution.
Sections in the Chinese source: 核心公式, 复现边界. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Basics
What can Switch Attention's local and full paths read?
Switch Attention: Towards Dynamic and Fine-grained Hybrid Transformers
Show answer and example
Answer: Local reads the visible window; full reads all causal history, never the future. The router trades evidence range and compute per token.
Illustrative example: A nearby phrase may need only local context, while an old fact needs a wider read.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Core mechanism
Why does a steady-state worker pool use compute better than waiting for every experiment in a generation?
AIRA2: Overcoming Bottlenecks in AI Research Agents
Show answer and example
Answer: Experiment lengths differ, so a barrier makes quick workers wait for the slowest. A steady pool assigns the next candidate immediately. Throughput must not alter evaluation isolation: search uses fixed search scores, then untouched validation picks a winner afterward. Do not peek mid-search.
Illustrative example: Example: Three experiments finish in ten minutes and one takes an hour. Reuse idle slots while keeping the predefined data splits.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Implementation details
Can 80/10/10 be randomly resplit for every candidate?
AIRA2: Overcoming Bottlenecks in AI Research Agents
Show answer and example
Answer: No. One split must be shared so data difficulty does not confound score comparisons. Local tests verify asynchronous scheduling and signal isolation, not multi-H200, Apptainer, or long-running research agents. Rank-based selection probabilities are search rules, not actual performance.
Illustrative example: Example: Identical models can rank differently if one gets easier search samples. A fixed split controls that difference.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Method comparison
Why separate the search split and final validation?
AIRA2: Overcoming Bottlenecks in AI Research Agents
Show answer and example
Answer: Search repeatedly uses scores and can overfit by hill climbing. Untouched validation chooses a winner only after search, reducing selection overfitting. Neither is training data, and repeatedly tuning search on validation undermines it. Hidden test stays out of the interface.
Illustrative example: Example: Explore 100 candidates by search ranking, then use the held-out validation once to choose the final model rather than inspect it after every proposal.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Basics
Why is automated ML research also an experiment scheduling problem in AIR-A2?
AIRA2: Overcoming Bottlenecks in AI Research Agents
Show answer and example
Answer: Agents can propose many candidates, but experiments vary in runtime and devices and evaluation are finite. Proposal quality and execution throughput both affect progress. AIR-A2 keeps experiments flowing and manages feedback. Busy workers do not establish research gains; verify selected proposals on isolated evaluation.
Illustrative example: Example: Waiting for a two-hour experiment can leave devices idle after a ten-minute one finishes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why expand exposure-external data and train hierarchically rather than only grow the model?
UniScale: Synergistic Entire Space Data and Model Scaling for Search Ranking
Show answer and example
Answer: More parameters can overfit limited exposure. Broader data expands coverage and hierarchical fusion treats heterogeneous quality appropriately. Verify synthetic/unexposed reliability; more data need not reduce bias.
Illustrative example: Add unseen categories without giving weak labels the same weight as verified clicks.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Does local UniScale gated FFN represent the full ES³ data pipeline?
UniScale: Synergistic Entire Space Data and Model Scaling for Search Ranking
Show answer and example
Answer: No. It tests heterogeneous branches, not private full-space sampling, production models, or cheap deployment. Audit source labels, splits, and mixing weights.
Illustrative example: Working FFNs do not prove reliable exposure-external data generation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
Are unexposed examples equivalent to random negatives?
UniScale: Synergistic Entire Space Data and Model Scaling for Search Ranking
Show answer and example
Answer: No. Sources and label reliability differ. Not exposed is not disliked, and full-space expansion does not automatically remove selection bias.
Illustrative example: A user who never saw an item cannot be assumed not to click it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why expand both data space and model capacity?
UniScale: Synergistic Entire Space Data and Model Scaling for Search Ranking
Show answer and example
Answer: Larger models still face exposure bias; more data can also exceed current capacity. ES³ expands beyond exposure, HHSFT fuses sources, and capacity must remain deployable.
Illustrative example: Adding unexposed examples differs from adding layers to old exposure data.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why does GateSID let cold-start degree control semantic–collaborative fusion and alignment strength?
GateSID: Adaptive Gating for Semantic-Collaborative Alignment in Cold-Start Recommendation
Show answer and example
Answer: Sparse feedback makes collaborative vectors noisy, so strong alignment can corrupt content. Rich feedback contains distinctions that content averaging may erase. Condition both fusion and alignment on evidence strength, adapting as interactions arrive instead of assigning all new items permanently to content only.
Illustrative example: Two textually similar items differ because one has strong audience feedback while the other just launched; a fixed 50:50 mix ignores that.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
Why must GateSID's cold-start features use only pre-cutoff data?
GateSID: Adaptive Gating for Semantic-Collaborative Alignment in Cold-Start Recommendation
Show answer and example
Answer: Counts and activity set the gate. Test-future interactions reveal later maturity and corrupt the cold-start scenario. Align dimensions before fusion and use real per-example adaptation rather than calling a fixed validation constant adaptive.
Illustrative example: An item that goes viral during test cannot use the week's final click count for an earlier recommendation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
How does gating alignment strength in GateSID differ from mixing final scores?
GateSID: Adaptive Gating for Semantic-Collaborative Alignment in Cold-Start Recommendation
Show answer and example
Answer: Final blending only changes use of existing representations. Training-time alignment changes how closely the spaces are learned together. Strong alignment can propagate noisy evidence, so conditional gating limits it; the mechanism does not prove gains in all cold-start slices.
Illustrative example: Lowering a new item's ID score at the end may not undo semantic corruption caused by forcing early alignment to a noisy ID.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why use cold-start degree to set semantic and collaborative proportions in GateSID?
GateSID: Adaptive Gating for Semantic-Collaborative Alignment in Cold-Start Recommendation
Show answer and example
Answer: Content shares knowledge when collaborative feedback is scarce; mature ID behavior should preserve distinctions. A state-conditioned gate avoids one ratio across items but activity is not the only reliability signal and can mislead.
Illustrative example: A new running shoe borrows content, while an established one keeps audience-specific behavior despite similar descriptions.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why build category manuals before distilling multimodal tagging to a small model?
TagLLM: A Fine-Grained Tag Generation Approach for Note Recommendation
Show answer and example
Answer: Manuals define category-specific concepts that broad tags miss. A larger model supplies rich supervision and staged distillation enables cheap serving. Omitted attributes are not automatically learned.
Illustrative example: Food needs flavor/method and travel needs destination/transport rather than treating “relaxed” identically.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
Does MovieLens verify multimodal note tagging?
TagLLM: A Fine-Grained Tag Generation Approach for Note Recommendation
Show answer and example
Answer: No. Genre and behavior proxies test mechanisms without private images, manual generation, or production models. Teacher context H is supervision only; exclude test labels from construction.
Illustrative example: Genre prediction is not fine-grained multimodal reasoning.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
How does two-stage distillation differ from calling a large model per note?
TagLLM: A Fine-Grained Tag Generation Approach for Note Recommendation
Show answer and example
Answer: It transfers expensive labeling to a smaller student, reducing serving cost subject to student capacity and coverage. Cached labels alone are not training.
Illustrative example: A student must label unseen notes, not only look up old examples.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why use category-specific interest manuals for TagLLM?
TagLLM: A Fine-Grained Tag Generation Approach for Note Recommendation
Show answer and example
Answer: Different content categories need different fine-grained attributes. Manuals and multimodal reasoning structure extraction; richer labels still need user validation.
Illustrative example: Food notes can encode taste and cooking method, not merely “food.”
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why normalize advantages within groups and across tasks?
SaFRO: Satisfaction-Aware Fusion via Dual-Relative Policy Optimization for Short-Video Search
Show answer and example
Answer: Within-group ranking identifies better candidates for one objective; cross-task terms compare objectives for the same candidate. Relation gates organize search objectives, limiting domination by high-scale rewards without defining long-term value or guaranteeing joint gains.
Illustrative example: Normalize dense clicks and sparse retention separately before balancing them.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Does local SAFRO on MovieLens reproduce short-video retention?
SaFRO: Satisfaction-Aware Fusion via Dual-Relative Policy Optimization for Short-Video Search
Show answer and example
Answer: No. Public full-catalog ranking tests proxy calculations, not private search, retention, or live buckets. Preserve negative results and do not call same-seed gaps online uplift.
Illustrative example: Offline movie hits are not next-day visits.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
Does standardization determine the relative importance of engagement and retention?
SaFRO: Satisfaction-Aware Fusion via Dual-Relative Policy Optimization for Short-Video Search
Show answer and example
Answer: No. It aligns scales, not business values. Relation gating combines goals without turning proxies into causal value. Specify reward definitions, axes, and fusion separately.
Illustrative example: Standardizing two exams does not determine their graduation weights.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Does each SAFRO rollout have one reward or several task rewards?
SaFRO: Satisfaction-Aware Fusion via Dual-Relative Policy Optimization for Short-Video Search
Show answer and example
Answer: rᵢ,ₜ indexes candidate and task separately. A candidate can score differently on immediate interaction and long-term signals; preserve that matrix for comparisons.
Illustrative example: Do not treat two objectives on one item as two items on one objective.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Core mechanism
Why jointly optimize GenFacet facet generation and query rewriting on downstream retrieval feedback?
GenFacet: End-to-End Generative Faceted Search via Multi-Task Preference Alignment in E-Commerce
Show answer and example
Answer: Plausible facet text can yield poor search, and rewrites must preserve intent under the selected facet. Multitask preference alignment uses facet quality and actual retrieval results to avoid mismatched local goals. Feedback can favor popular facets; fluency or one hit metric is insufficient.
Illustrative example: A lightweight-laptop weight facet should yield a searchable weight constraint, not a fluent rewrite about performance rankings.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Implementation details
Why preserve facet-to-rewrite correspondence in GenFacet training?
GenFacet: End-to-End Generative Faceted Search via Multi-Task Preference Alignment in E-Commerce
Show answer and example
Answer: Retrieval credit belongs to the context and facet that produced that query. Define task weights, reference policy, and valid-output masks. Deterministic candidate scores without actual generation and updates do not establish teacher distillation plus GRPO.
Illustrative example: Credit a successful waterproof query to its own facet, not a lightweight facet.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Method comparison
How does GenFacet multitask preference alignment differ from text-only teacher distillation?
GenFacet: End-to-End Generative Faceted Search via Multi-Task Preference Alignment in E-Commerce
Show answer and example
Answer: Text distillation copies teacher wording. Multitask rewards score the facet and retrieved results, correcting fluent but useless choices while potentially neglecting unmeasured constraints. Negative local results cannot be replaced by production design goals.
Illustrative example: Machine washable may retrieve useful items where an elegant lifestyle phrase does not, but easy keyword hits can also game the reward.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Basics
Why treat GenFacet facet selection and query rewriting as one search pipeline?
GenFacet: End-to-End Generative Faceted Search via Multi-Task Preference Alignment in E-Commerce
Show answer and example
Answer: A facet determines the refinement, and the rewrite turns it into a retrievable query. Independent optimization can produce attractive facets with no results or drift from intent. Joint retrieval feedback depends on catalog and reward reliability.
Illustrative example: Commuting can refine a query toward light, waterproof products, not specialist off-road equipment.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why keep Direct and Reasoning query paths?
AIGQ: An End-to-End Hybrid Generative Architecture for E-commerce Query Recommendation
Show answer and example
Answer: Easy intent benefits from low latency and complex intent from deeper inference. IL-GRPO aligns query groups, while hybrid offline/online serving reduces request cost. Local proxies do not reproduce CTR rewards or the live stack.
Illustrative example: Directly complete a brand, but reason about recipient and use for vague gifts.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Does MovieLens validate live IL-GRPO and latency?
AIGQ: An End-to-End Hybrid Generative Architecture for E-commerce Query Recommendation
Show answer and example
Answer: No. Same candidates/seeds and validation-tuned proxy weights test ranking mechanisms only, without real query CTR or hybrid serving. Account separately for ranking, policy training, and measured latency.
Illustrative example: Movie NDCG gains do not establish millisecond query latency or online RL.
Sections in the Chinese source: 本地复现. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
What update risk does KL control beyond CTR reward?
AIGQ: An End-to-End Hybrid Generative Architecture for E-commerce Query Recommendation
Show answer and example
Answer: Reference KL penalizes drifting into narrow or reward-hacking queries. It balances feedback against prior behavior without guaranteeing unbiased rewards or independent quality.
Illustrative example: Avoid making every suggestion clickbait just because clicks rise temporarily.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Why treat suggested queries as a set rather than independent strings?
AIGQ: An End-to-End Hybrid Generative Architecture for E-commerce Query Recommendation
Show answer and example
Answer: A group defines exploration coverage. Per-item plausibility can yield repetitive outputs; group rewards evaluate coverage and engagement, subject to reward quality.
Illustrative example: Five paraphrases add less than options covering director, mood, and duration.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why are versioned skills more reusable than appending raw execution logs to context?
Show answer and example
Answer: A log describes one run. Structured skills extract applicability and steps for later retrieval. Versioned merge tracks corrections instead of keeping contradictory old copies. Abstraction must preserve provenance and failure conditions; shorter wording does not make it a reliable general procedure.
Illustrative example: Example: A deployment log contains temporary directories and retry noise. The skill retains dependency checks and rollback conditions, and a later version fixes obsolete arguments.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
What does low local joint success with nonzero skill telemetry show?
Show answer and example
Answer: The mechanism ran and many tasks still failed. Creation or reuse counts cannot be cherry-picked as improvement. Local tests cover state updates and fair budgets without the original large model, real web, or full benchmark. Keep negative outcomes with process evidence.
Illustrative example: Example: Ten skill retrievals show invocation happened. If the task failed, they do not establish correct content or execution benefit.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
How does versioned Merge differ from appending a new skill every time?
Show answer and example
Answer: Merge integrates feedback into relevant existing skills and records changes rather than accumulating near duplicates. Appending is simpler but can retain contradictory versions. Merge must preserve applicability and check regressions; one topical or successful example does not justify a global replacement.
Illustrative example: Example: Update an export skill for a pagination limit. If it applies to one service only, keep that condition rather than requiring pagination everywhere.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
What repeated work does Memento-Skills avoid across similar tasks?
Show answer and example
Answer: A run records useful steps, errors, and conditions. Raw logs still need interpretation next time. Memento-Skills makes experience searchable and updatable for reuse. Usefulness depends on applicability and actual execution; storage and version counts do not measure capability.
Illustrative example: Example: After discovering a billing date-format issue, reuse a skill with format validation instead of rereading the entire failure log.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Core mechanism
Why does parent sampling consider both performance and existing offspring count?
Show answer and example
Answer: Selecting only top scores repeatedly expands the same branch and reduces diversity. Penalizing offspring count gives viable, less explored parents opportunities. Both the task agent and its modification mechanism are editable, allowing the improvement process itself to evolve. Sampling probability allocates budget; it does not accept every child or guarantee monotonic gains.
Illustrative example: A champion with twenty children may get fewer chances than a slightly weaker parent tried once, opening other improvement paths rather than polishing one branch indefinitely.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Implementation details
Does the local archive step execute model-generated source or shell commands?
Show answer and example
Answer: No. It calculates parent distributions and maintains a bounded structured archive. Generated code and shell execution are prohibited, so the path does not validate self-referential program modification. Probabilities summing to one and correct lineage updates are search-control evidence, not task- and meta-agent behavior evaluation.
Illustrative example: Editing one field tests the archive, but without running the changed program we cannot know whether it breaks the task or improves search.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Method comparison
How does fixing the meta-agent versus making it editable change the search space?
Show answer and example
Answer: A fixed improver generates variants only in the task-solution space. Editing the meta-agent can also change candidate generation and improvement procedures. The extra freedom adds failure and attribution challenges, requiring separate execution and evaluation. Local archive operations generate or run no arbitrary source, so lineage is not full program self-modification.
Illustrative example: Changing a sorting function is task-level. Changing how functions are selected for trials is meta-level, and the extra freedom may not help.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Basics
Why does HyperAgents edit both task solutions and the program that generates the next solution?
Show answer and example
Answer: A fixed improver searches with the same proposal and reflection routines, whose limitations can block progress. HyperAgents makes the task and meta programs editable to study whether improvement methods evolve through experience. Self-modifying code is harder to validate; a new archived version does not establish actual performance or sustained gains.
Illustrative example: If one version always suggests similar changes, try editing the proposer itself. Then verify that the new proposer actually produces more effective candidates.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why does GLIDE use separate prompts for recent behavior and long-term user embeddings?
Deploying Semantic ID-based Generative Retrieval for Large-Scale Podcast Discovery at Spotify
Show answer and example
Answer: Recent behavior captures immediate intent but can repeatedly favor familiar items. The long-term representation adds stable preferences missed by a short window. Both condition SID generation, trading responsiveness and exploration; long-term features can go stale and do not establish exploration by themselves.
Illustrative example: Recent commute podcasts can inform context while stable science interest helps discover a new science show.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Why must GLIDE's long-term embedding exclude the complete post-test user history?
Deploying Semantic ID-based Generative Retrieval for Large-Scale Podcast Discovery at Spotify
Show answer and example
Answer: Both long-term summaries and recent sequences must stop at available behavior. A test target inside the long-term vector leaks even if absent from recent inputs. Document codebook fitting and the local three-level public proxy.
Illustrative example: A movie's vector cannot contribute to the history used to predict that same held-out movie.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
How does GLIDE's dual-prompt generation change outputs compared with transition or content retrieval?
Deploying Semantic ID-based Generative Retrieval for Large-Scale Podcast Discovery at Spotify
Show answer and example
Answer: Traditional systems directly combine similarity or transition scores. GLIDE predicts conditioned SID-path probabilities and maps them to candidates, sharing semantics through discrete codes while adding quantization and early-path errors. A local code-transition proxy does not train the full language generator.
Illustrative example: Generate a photography coarse code then refine it. Choosing the wrong branch can exclude a target that full-catalog scoring could still rank.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Why separate GLIDE's recent behavior and long-term embeddings into two prompt types?
Deploying Semantic ID-based Generative Retrieval for Large-Scale Podcast Discovery at Spotify
Show answer and example
Answer: The sequence retains immediate details; the embedding summarizes stable preferences. Separate inputs avoid averaging them together. Hierarchical SID paths still depend on prompt and codebook coverage; their combination does not prove reduced filter bubbles.
Illustrative example: Preserve long-term photography and recent camping signals to support outdoor cameras or campsite gear.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Core mechanism
Why doesn't mechanical transliteration determine whether knowledge exists?
Large Reasoning Models Struggle to Transfer Parametric Knowledge Across Scripts
Show answer and example
Answer: Script changes can affect identity recognition, retrieval and expression, which before/after outputs cannot disentangle. The paper uses regression, training entity prompts and actual LoRA. Local reference notes are not capability.
Illustrative example: Failure on a name in another script may be entity mismatch; hints diagnose but don't establish transfer training.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Implementation details
Is there an executable local knowledge transfer operator?
Large Reasoning Models Struggle to Transfer Parametric Knowledge Across Scripts
Show answer and example
Answer: No. Literature only; real LM semantics and LoRA training are not implemented. Equations/generic SFT don't imply reproduction, and prompts must not include test answers.
Illustrative example: A working transliteration function is not the paper's parametric retrieval mechanism.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Method comparison
Can inference transliteration replace training-time entity interventions?
Large Reasoning Models Struggle to Transfer Parametric Knowledge Across Scripts
Show answer and example
Answer: No. Training interventions study retrieval/learning pathways; transliteration changes current input and may bypass difficulty. String rules or a small classifier cannot stand in for LoRA and semantic transfer.
Illustrative example: Normalizing names to a familiar script can help without teaching cross-script transfer.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Basics
Why does cross-script knowledge research test more than translation?
Large Reasoning Models Struggle to Transfer Parametric Knowledge Across Scripts
Show answer and example
Answer: Another script can affect entity recognition, retrieval of parametric facts and reasoning expression. Translation is one factor. Regression/interventions separate them; this isn't a new RL algorithm.
Illustrative example: Failing on a transliterated name doesn't prove the model never learned facts about that person.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Core mechanism
Why maintain a history-based mean and use a semantic epsilon-net to limit redundancy?
POLCA: Stochastic Generative Optimization with LLM
Show answer and example
Answer: Random data and evaluators introduce noise, so one high score may be accidental. Repeated observations stabilize candidate comparisons. Semantic distance constraints prevent budget waste on near-duplicates; recompute distances after adding candidates. Keep empirical-mean rules distinct from theoretical UCB, and don't interpret rankings as statistical significance.
Illustrative example: Semantic filtering can avoid evaluating ten prompts differing only in punctuation. Combine a lucky first score with later evaluations rather than making it a permanent champion.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Implementation details
How are untested proposals and observations completed before failure recorded after the metric budget is exhausted?
POLCA: Stochastic Generative Optimization with LLM
Show answer and example
Answer: Charge each actual call before making it and keep observations completed before failure. Mark unevaluated proposals deferred_budget and omit them from observed-reward statistics. Old and new candidates use the same with-replacement batch. There is no mock LLM fallback; deep-copy callback inputs and outputs to protect history. Local tests do not demonstrate real generative optimization.
Illustrative example: If the last budget evaluates one new proposal, the other three are neither zero scores nor default successes. They have no observations and must remain separate from evaluated candidates.
Sections in the Chinese source: 核心公式, 本地复现, 代码映射与边界. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Method comparison
Why distinguish empirical-mean priority in the POLCA main text from the theoretical UCB variant?
POLCA: Stochastic Generative Optimization with LLM
Show answer and example
Answer: Empirical mean ranks candidates by their observed average reward; UCB explicitly adds an exploration bonus for uncertainty. They can prioritize rarely evaluated candidates differently and change budget use and selection history. Describe implementation and theoretical rules separately; properties do not transfer to an unimplemented variant by name.
Illustrative example: A candidate evaluated once might not rank high by mean. UCB can give it another trial because uncertainty is high, producing a different next round.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Basics
Why does noisy optimization easily mistake a lucky high score for a good solution?
POLCA: Stochastic Generative Optimization with LLM
Show answer and example
Answer: Random batches, execution, and evaluation vary the score of one candidate. Keeping only observed maxima favors luck. POLCA uses evaluation history and global experience while limiting redundant proposals. More history does not eliminate bias: use consistent evaluation and budgets, and do not replace actual observations with expected outcomes of untested proposals.
Illustrative example: Configuration A may score high on an easy batch while B is stable across batches. Comparing only maxima can choose wrongly; keep full histories to check.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why align retrieval sets before compiling into a diffusion fan-out retriever?
Efficient, Property-Aligned Fan-Out Retrieval via RL-Compiled Diffusion
Show answer and example
Answer: Set value includes coverage and diversity beyond item relevance. Define trade-offs with a composite set reward, then train fast diffusion retrieval to approximate it. Reward alignment may be imperfect; local work only implements the set objective.
Illustrative example: Three individually relevant duplicates can be worse than a set covering separate facts.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
Implementation details
Does the local R4T component train the 53.9M diffusion retriever?
Efficient, Property-Aligned Fan-Out Retrieval via RL-Compiled Diffusion
Show answer and example
Answer: No. It computes the alignment/diversity objective, without training the full generator. Scoring candidates is not generating fan-out results.
Illustrative example: A function ranking supplied sets cannot necessarily retrieve a set from a query.
Sections in the Chinese source: 本地复现, 复现边界. The example illustrates the mechanism.
Method comparison
How does set alignment differ from independent item ranking?
Efficient, Property-Aligned Fan-Out Retrieval via RL-Compiled Diffusion
Show answer and example
Answer: Sets account for relations and redundancy. Summing independent scores can miss coverage. The compiled retriever requires separate training and evaluation.
Illustrative example: Three high-relevance duplicates can have low diversity.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
Basics
Why build a set-level alignment target before training a fast retriever?
Efficient, Property-Aligned Fan-Out Retrieval via RL-Compiled Diffusion
Show answer and example
Answer: Item scores alone may not guarantee whole-set properties. Compile a composite set target into diffusion fan-out to seek fewer online steps.
Illustrative example: Relevant outputs should cover different evidence rather than repeating one top item.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why co-design tokenization, attention, and FFNs?
SORT: A Systematically Optimized Ranking Transformer for Industrial-scale Recommenders
Show answer and example
Answer: Heterogeneous fields and long histories need appropriate token interfaces, information flow, and tokenwise capacity. Compression changes resource allocation. Local mechanisms do not establish full DLRM replacement throughput or quality.
Illustrative example: Organize age, category, and history rather than treating them all as ordinary text tokens.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Does local residual MHA/FFN demonstrate industrial compression?
SORT: A Systematically Optimized Ranking Transformer for Industrial-scale Recommenders
Show answer and example
Answer: No. Fixed public candidates exercise core paths without private features, production scale, or kernels. Model structure alone is not measured QPS or uplift.
Illustrative example: A small catalog run does not validate latency over industrial sparse tokens.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does a ranking Transformer differ from a traditional DLRM?
SORT: A Systematically Optimized Ranking Transformer for Industrial-scale Recommenders
Show answer and example
Answer: Token interaction handles features and histories; DLRMs commonly combine embeddings, explicit interactions, and MLPs. Cost and feature processing affect deployment, not names alone.
Illustrative example: The same IDs can interact at different stages with different compute reuse.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why isn't SORT Ranking simply a language Transformer for recommendations?
SORT: A Systematically Optimized Ranking Transformer for Industrial-scale Recommenders
Show answer and example
Answer: Industrial fields, tokenization, attention, FFNs, and hardware layout are co-designed to replace DLRM. Generic residual equations are not the full system.
Illustrative example: Sparse categorical features are not prepared like natural-language words.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Core mechanism
Why harmonize HAP scores after routing difficult candidates to the strong branch?
Not All Candidates are Created Equal: Heterogeneity-Aware Pre-ranking
Show answer and example
Answer: Branch compute differs and raw score distributions may not be comparable. Difficulty routing allocates work and harmonization aligns rankings across branches. Misrouting can still lose key items, so measure ranking quality alongside compute.
Illustrative example: A light branch averaging 0.8 would unfairly dominate a strong branch averaging 0.4 without scale alignment.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Implementation details
Why distinguish soft HAP training from actual compute budgets?
Not All Candidates are Created Equal: Heterogeneity-Aware Pre-ranking
Show answer and example
Answer: Soft mixtures can execute both branches even when a gate is near zero or one. Hard serving routes need explicit thresholds and fallbacks; local ranking proxies do not measure production latency.
Illustrative example: Running both networks then multiplying by a binary gate changes output but saves no FLOPs; verify the inactive branch actually does not run.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Method comparison
What cross-path relation does HAP harmonization address beyond calibrating only the strong branch?
Not All Candidates are Created Equal: Heterogeneity-Aware Pre-ranking
Show answer and example
Answer: Items from both paths compete in one ranking; internal accuracy is insufficient. Harmonization aligns score scales and preferences across budgets to prevent route identity becoming an advantage. It can limit specialization, so measure cost and quality.
Illustrative example: A simple candidate scored 0.8 should not automatically beat a difficult valuable one scored 0.4 by a different branch.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Basics
Why does HAP allocate light or strong pre-ranking by candidate difficulty?
Not All Candidates are Created Equal: Heterogeneity-Aware Pre-ranking
Show answer and example
Answer: Easy negatives need little compute, while ambiguous candidates benefit from more interactions. Routing has cost and can err, sending important items to the light branch. Dynamic budgeting is not simply extra work for popular items.
Illustrative example: Quickly reject clearly irrelevant products and compare two nearly identical eligible products with the strong model.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
How do CGR's decoding-time ad-load constraints differ from deleting surplus ads afterward?
Constraint-Aware Generative Re-ranking for Multi-Objective Optimization in Advertising Feeds
Show answer and example
Answer: Post-hoc deletions change context and remaining slots, invalidating value estimates for the original prefixes. CGR maintains the feasible set at each generation step and combines it with estimated reward to prune infeasible branches. A high-scoring but unusable list is not a valid result.
Illustrative example: With at most two ads in five positions, mask ads after using both slots. Generating four ads and deleting two changes the list the system scored.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
When should an implementation of CGR update `F(y<t)`?
Constraint-Aware Generative Re-ranking for Multi-Objective Optimization in Advertising Feeds
Show answer and example
Answer: After each selection, update the prefix, load, and position state, then construct the next feasible set. Do not generate the whole sequence and assume every prefix was legal. An empty set must terminate or use a defined legal fallback, never silently fill with an invalid item.
Illustrative example: If the final slot has no eligible organic items and the ad cap is reached, report or terminate as defined instead of adding an ad.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
How does CGR's constraint-aware pruning differ from fixed item ranking in prefix dependence?
Constraint-Aware Generative Re-ranking for Multi-Objective Optimization in Advertising Feeds
Show answer and example
Answer: Fixed item scores do not depend on selected items. CGR updates feasibility and reward at each step, so count and position rules directly change later choices. This reduces repairs but requires accurate value estimates and search budget; one proxy comparison does not prove global optimality.
Illustrative example: An ad in slot one can force organic content in slot two under spacing rules without changing any ordinary item score.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
What decisions does CGR unify by putting generation and reward estimation in one network?
Constraint-Aware Generative Re-ranking for Multi-Objective Optimization in Advertising Feeds
Show answer and example
Answer: The model proposes the next item and estimates its business value after insertion, conditioned on the prefix, ad positions, and load rules. Feasibility determines what can be chosen; reward compares legal options. A high reward does not override the rules.
Illustrative example: Once the screen reaches its ad cap, even a high-value ad cannot be generated; rank eligible organic items instead.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
Why calculate group mean and variance from raw rewards while applying successful-response length penalty only in the advantage numerator?
Show answer and example
Answer: Length penalty modulates credit for one successful response, not the group's baseline and scale. Attacked groups also combine raw attacker and defender returns. Penalizing before whitening changes their comparison and the algorithm. Exclude zero-variance groups rather than using arbitrary epsilon to manufacture signal.
Illustrative example: Two equally successful responses differ in length. Penalize the longer advantage without letting extra text drag the common raw-return baseline.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
Should DMAST's training-to-sampling probability ratio multiply both PG and KL?
Show answer and example
Answer: No. Only PG receives min(π_old/π_sampling,1), and PG is zero if the ratio is below 0.1. KL is not scaled by that condition. Exclude zero-variance groups and normalize by all valid response tokens in the batch rather than averaging per role.
Illustrative example: Dropping PG for a low-ratio token does not justify deleting its reference KL.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
What is missing if only DMAST's group-statistics objective is implemented?
Show answer and example
Answer: It can verify loss and gradients. Full visual-agent adversarial training needs a real VLM, environment interaction, teacher imitation, oracle SFT, and self-play. A mathematical diagnostic generates no attack payload and does not establish browser success or robustness.
Illustrative example: One SGD step on a tiny policy cannot support the same capability claim as all training stages in an interactive environment.
Sections in the Chinese source: 背景与主要改动, 本地复现. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why does DMAST share VLM parameters across agent and attacker roles?
Show answer and example
Answer: Role conditioning supports interaction and self-play with shared parameters, so robustness training covers defense decisions and attack trajectories. The full method also uses teacher imitation and oracle-reasoning SFT; the Stage 3 reward formula is one part.
Illustrative example: One base model generates both kinds of trajectories under different role prompts, not two independent policies.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why add Pin–Board neighbor alignment beyond image–text alignment in PinCLIP?
PinCLIP: Large-scale Foundational Multimodal Representation at Pinterest
Show answer and example
Answer: Generic image–text alignment captures content compatibility, while user boards expose platform-specific relationships. Neighbor supervision adds graph semantics and multigranular features retain details for new content. Exposure bias can persist, so graph alignment is not unbiased preference estimation.
Illustrative example: Illustration: A table and a lamp may look different but share Scandinavian décor boards, letting a new lamp connect to related furniture.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
Why exclude future test edges from PinCLIP graph training rather than just freeze the image encoder?
PinCLIP: Large-scale Foundational Multimodal Representation at Pinterest
Show answer and example
Answer: Future consumption edges can encode target relations even without encoder updates. Graph construction, negatives, and new-content splits need cutoffs. Distinguish local alignment from original pretraining; vector shape alone does not establish fidelity.
Illustrative example: Illustration: Adding a future saved lamp edge exposes the answer before prediction regardless of frozen parameters.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
Where do PinCLIP positives come from in neighbor alignment versus image–text contrastive learning?
PinCLIP: Large-scale Foundational Multimodal Representation at Pinterest
Show answer and example
Answer: Image–text positives link modalities of one item; graph neighbors link distinct items on boards, learning pairing and interest structure. Signals can conflict, requiring weights and sampling. A sparse genre proxy cannot reproduce VLM distinctions, so its negative result is bounded.
Illustrative example: Illustration: A lamp's image and description are one positive pair; the lamp and a table on one board are another, not the same item.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why connect new PinCLIP content to the recommendation graph through content vectors?
PinCLIP: Large-scale Foundational Multimodal Representation at Pinterest
Show answer and example
Answer: New items lack collaboration. Existing board neighbors teach platform semantics and the content encoder transfers them to unseen items. Fine image–text detail helps, but graph exposure bias remains. Evaluate cold start separately.
Illustrative example: Illustration: An unclicked lamp can connect to décor-board furniture through content; that is not an observed co-purchase.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why align OneRanker's generation distribution and value head preferences?
OneRanker: Unified Generation and Ranking with One Model in Industrial Advertising Recommendation
Show answer and example
Answer: Generation controls candidate availability and value controls ranking afterward. Reranking cannot recover valuable items the generator never produces. A shared fake-item token aligns interfaces and a consistency loss aligns distributions. Balance generation fit, value accuracy, and consistency to avoid suppressing useful differences.
Illustrative example: Illustration: If generation favors cheap accessories while value prefers durable tools, reranking accessories cannot show tools. Consistency aims to give tools a generation chance.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Why not subtract raw pgen and prank logits for OneRanker's KL?
OneRanker: Unified Generation and Ranking with One Model in Industrial Advertising Recommendation
Show answer and example
Answer: KL compares normalized probabilities over aligned candidates and consistent temperatures. Mask padding and illegal candidates identically; do not expose value labels to generation. Fixed score mixing does not demonstrate learned distribution alignment.
Illustrative example: Illustration: A constant logit shift leaves probabilities unchanged but changes squared differences. Full-catalog and ten-item normalizations cannot be treated as the same KL.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
What does generation consistency KL enforce beyond a shared backbone?
OneRanker: Unified Generation and Ranking with One Model in Industrial Advertising Recommendation
Show answer and example
Answer: Different heads can learn opposing preferences despite shared parameters. KL aligns distributions over identical candidates. This may reduce conflict or constrain useful specialization, so calibrate it. Agreement alone does not improve recommendation; both may prefer the wrong popular items.
Illustrative example: Illustration: A backbone does not stop accessory generation and tool value rankings from disagreeing. KL penalizes that disagreement, with direction still mattering.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
What prediction interfaces does OneRanker's fake item token unify?
OneRanker: Unified Generation and Ranking with One Model in Industrial Advertising Recommendation
Show answer and example
Answer: A shared representation conditions generation likelihood and value ranking, reducing objective separation. The token is an architectural placeholder, not a real item. Map outputs to a legal catalog and preserve each head's objective.
Illustrative example: Illustration: One candidate state answers likelihood and expected value, but the placeholder itself cannot be served as an ad.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why does IDProxy align the MLLM and ID spaces before injecting multilayer proxy residuals into a ranker?
IDProxy: Cold-Start CTR Prediction for Ads and Recommendation at Xiaohongshu with Multimodal LLMs
Show answer and example
Answer: Multimodal semantics describe content but need not match the collaborative coordinates learned from long-term CTR training. Replacing IDs outright can destroy that geometry. Coarse contrastive alignment establishes a correspondence, then multilayer proxies extract knowledge at different abstraction levels and a gate controls its contribution. A residual protects the original path, but controlled comparisons still matter: overly strong semantics can swamp useful collaborative distinctions.
Illustrative example: Illustration: Two visually similar products may have different audiences, so semantics cannot simply override ID preferences. First align the spaces, then borrow attributes such as shape or use at appropriate levels.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
Why must IDProxy construct coarse-alignment pairs using the same item identity?
IDProxy: Cold-Start CTR Prediction for Ads and Recommendation at Xiaohongshu with Multimodal LLMs
Show answer and example
Answer: PAL brings an MLLM vector zi near its corresponding ID vector ei, with other IDs in the batch contributing to the denominator. Misaligned columns force unrelated items together. The residual adapter must project to the original ID dimension before addition, and the gate must belong to that candidate. Collect multilayer hidden states from the actual model version rather than substituting random vectors.
Illustrative example: Illustration: Camera content paired with a children's clothing ID can still reduce training loss while learning the wrong mapping. Matching shapes and plausible values cannot replace identity checks.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
What fallback does IDProxy's residual gate preserve relative to completely replacing item IDs?
IDProxy: Cold-Start CTR Prediction for Ads and Recommendation at Xiaohongshu with Multimodal LLMs
Show answer and example
Answer: Replacement discards existing collaborative memory; the gate adds adapted semantics on top of the original ID and can limit an unreliable increment. It can still favor one side too much, so check cold and tail items separately from head items. A lower contrastive pretraining loss does not guarantee better ranking; the source supports only its small public protocol.
Illustrative example: Illustration: An established item with abundant feedback may need little semantic input. A new item with a weak ID can borrow more content, but a gate cannot guarantee detection of every content error.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why is an IDProxy proxy not just the original MLLM embedding?
IDProxy: Cold-Start CTR Prediction for Ads and Recommendation at Xiaohongshu with Multimodal LLMs
Show answer and example
Answer: The MLLM learns content semantics, whereas ranking IDs capture collaborative behavior; their directions and scales can differ. After coarse alignment and multilayer adaptation, a gated proxy increment supplements the original ID representation. A concatenation that happens to run is not evidence of successful space adaptation.
Illustrative example: Illustration: Similar-looking products can have distinct audiences that raw content embeddings miss but ID history preserves. The proxy should add information without replacing that distinction.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why does HPGR use masked session pretraining before selecting history with preference-guided sparse attention?
Beyond the Flat Sequence: Hierarchical and Preference-Aware Generative Recommendations
Show answer and example
Answer: Session structure captures relationships within a stretch of behavior. Learning it with masking avoids treating the entire long history as a flat sequence of interchangeable events. At fine-tuning time, the current preference selects related history so computation focuses on useful evidence rather than just cutting by recency. Sparse selection can still miss rare but decisive interests, so check selection coverage along with final predictions.
Illustrative example: Illustration: Last week's camera comparisons belong to one session and can be retrieved together for today's lens candidate. The latest unrelated purchase should not displace that organized history just because it is newer.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
How should HPGR keep padding and future targets out of its top-k attention?
Beyond the Flat Sequence: Hierarchical and Preference-Aware Generative Recommendations
Show answer and example
Answer: Score only eligible historical events, mask padding before top-k, and enforce the current-time cutoff. If there are fewer than k events, use the valid count instead of duplicating dummy records. Align session boundaries with the preference query; do not use the gold next item to create a condition unavailable at test time.
Illustrative example: Illustration: With three valid events and k=5, attention can read only those three. Unmasked zero-scored padding can beat real events with negative scores.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
What older evidence can HPGR use that a fixed recent window might miss?
Beyond the Flat Sequence: Hierarchical and Preference-Aware Generative Recommendations
Show answer and example
Answer: A recent window retains events by time. Preference-guided sparse selection picks events for the current intent, letting older relevant history compete for a limited budget. It depends on the selector: downstream layers cannot read omitted events, and hierarchical pretraining does not guarantee correct selection. The public proxy gains in the source apply only to its protocol.
Illustrative example: Illustration: If the latest browsing was for novels but the current candidate is a camera, the selector may find older photography history that the last ten events omit. A bad selector could also revive an outdated interest.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why does HPGR use masked session modeling to learn a behavioral hierarchy?
Beyond the Flat Sequence: Hierarchical and Preference-Aware Generative Recommendations
Show answer and example
Answer: Events in a session often share intent, while a change of session may reflect a shift in intent. Learning both levels gives the later preference selector a more meaningful state. A flat sequence based on adjacent positions can mix separate visits. Pretraining supervision comes from observed training data; test input must not reveal a target that should be unknown.
Illustrative example: Illustration: One visit focuses on tents and sleeping bags, the next on hiking boots. A session encoder can distinguish the two needs rather than treating every item as part of one continuous intent.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Core mechanism
Why isn't OS in the data-quality theory full RL post-training?
Show answer and example
Answer: OS is outcome-supervised regression on final predictions. The comparison is SFT on teacher intermediate states. Linear attention and covariance assumptions enable closed-form analysis, not policy sampling, advantage estimation, or policy gradients.
Illustrative example: Squared error on final weights differs from intermediate-state matching without becoming reward-driven policy optimization.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Implementation details
Why must autonomous rollout not receive true w*?
Show answer and example
Answer: Ground-truth weights belong only in training loss, not prediction inputs. Use public context for rollout, feed targets only to the explicit loss, and preserve initialization's nullspace in closed-form SFT.
Illustrative example: A predictor cannot receive the exact weights it must recover.
Sections in the Chinese source: 核心公式, 本地复现. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Method comparison
How do SFT and OS differ in information supplied?
Show answer and example
Answer: SFT supervises intermediate teacher weights. OS runs an autonomous rollout and supervises only its final prediction, without implying policy gradients.
Illustrative example: Stepwise standard results and a single final weight check are distinct signals.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Basics
Why use linear regression to analyze pre- and post-training?
Show answer and example
Answer: Controlled linear attention exposes how covariance, initialization, and supervision affect weight prediction and permits closed-form references. It is not empirical evidence for full LLM training.
Illustrative example: Change task covariance in a small matrix and isolate how initialization affects fitting.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why use business-eligibility-dependent collision margins in QuaSID instead of repelling every similar item equally?
Show answer and example
Answer: Nearby codes share semantics, but collisions can confuse items that must remain distinct in serving. Eligibility weights allocate separation where it matters, including sparse items, while uniform repulsion wastes capacity and breaks useful neighborhoods. Outcomes still depend on signal quality.
Illustrative example: Illustration: Nearly identical items may share a coarse code but need different fine codes for distinct delivery eligibility.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
Why stop pushing QuaSID distances after the margin?
Show answer and example
Answer: [mij−d]+ is zero beyond the margin: enough separation is the goal. Align qij and margin with the same pair and use training-visible eligibility. Fix distance and symmetry conventions; do not derive favorable pairs from test eligibility.
Illustrative example: Illustration: Always minimizing negative distance would repel already-separated codes and destroy useful semantic structure.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
How do differentiated QuaSID margins allocate code space compared with uniform collision repulsion?
Show answer and example
Answer: Uniform pressure can waste capacity and damage semantic neighborhoods. Eligibility-weighted separations prioritize business requirements, but depend on accurate and current eligibility. The public proxy regressed slightly; design specificity is not measured improvement.
Illustrative example: Illustration: Separate items with incompatible inventory eligibility, but erroneous eligibility could split otherwise useful substitutes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Does a larger QuaSID distance imply one item is preferred?
Show answer and example
Answer: No. Margin and eligibility indicate which pairs need distinguishable addresses, not user preference direction. Both items can be relevant or irrelevant. User-conditioned scoring still determines ranking.
Illustrative example: Illustration: Two delivery eligibility classes can separate cleanly without making either item desirable to the user.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why can't a GRC corrector read the first wrong SID position during evaluation?
Show answer and example
Answer: Gold codes only build training draft–reflection–correction supervision. Deployment must infer likely errors from candidates, attribute consistency, and uncertainty without answer leakage. Entropy allocates correction budget, but hard high-entropy cases need not improve.
Illustrative example: A training example can identify a wrong city code; live recommendation cannot know the user's eventual target code.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
How should GRC separate training first-error labels from test reflection inputs?
Show answer and example
Answer: Compare drafts with gold during training and use `L+1` if all levels match. At test, use only model drafts, attributes, and uncertainty. Bind reflection, correction, and reward to one trajectory without gold-based repairs.
Illustrative example: Giving the second-level gold mismatch to the test corrector makes it an answer-aware diagnostic, not autonomous repair.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
What does GRC trade by allocating correction to uncertainty rather than fixing every beam equally?
Show answer and example
Answer: Reflection-token entropy prioritizes likely repair candidates under a fixed budget, reducing duplicate work. High entropy can be noise and low entropy can be confidently wrong. The reported reward rose but test ranking fell; retain both.
Illustrative example: Correct two uncertain beams out of ten first, while recognizing that a confident wrong path might be missed.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Why structure GRC trajectories as draft, reflection, and corrected SID?
Show answer and example
Answer: Early hierarchical mistakes restrict later choices. Explicit steps teach locating and repairing deviations instead of only imitating correct codes. At inference the model must identify errors itself, and valid formatting does not guarantee effective correction.
Illustrative example: Switch a home-goods draft to cameras by recognizing the request evidence, not by seeing a gold first-error index.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why use full-vocabulary log-softmax before masking?
Show answer and example
Answer: Constraints remove invalid paths, not alter original conditional probabilities. Legal-set renormalization favors some small sets in cross-beam comparison. CSR accelerates lookup; synchronize prefixes and states.
Illustrative example: A lone legal child should not become probability one and unfairly beat a better-supported path.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Why reorder prefix and trie node together after beam selection?
Show answer and example
Answer: Prefix and legal state are paired. Children can come from different parents; mismatching states applies the wrong mask. Pin trie versions.
Illustrative example: A clothing prefix cannot keep a food node for next-step legality.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
How does legal-only softmax differ from full log-softmax then masking?
Show answer and example
Answer: Legal renormalization changes prefix score scales according to set size. Full log-softmax preserves model probabilities while excluding invalid tokens.
Illustrative example: A parent with one legal successor gets an artificial boost if renormalized alone.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Why compile legal item tries into device-side transitions?
Show answer and example
Answer: Inventory/geography constraints need legal decoding, but CPU pointer traversals cause round trips. CSR and shallow dense lookup provide contiguous token and destination indices on accelerators.
Illustrative example: Gather legal successors from one row rather than walking a CPU trie.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why use short SID prefixes and unique IDs instead of all-SID generation or global ANN?
SIGMA: A Semantic-Grounded Instruction-Driven Generative Multi-Task Recommender at AliExpress
Show answer and example
Answer: Prefix generation locates a semantic catalog; unique IDs disambiguate items and local ANN limits cost. Multiview alignment connects semantics and behavior; APF trades precision and diversity with confidence. Missed prefixes constrain recall.
Illustrative example: Search within outdoor equipment; if only home goods was generated, local ANN cannot find tents.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Does exact local cosine top-k establish production ANN latency?
SIGMA: A Semantic-Grounded Instruction-Driven Generative Multi-Task Recommender at AliExpress
Show answer and example
Answer: No. It verifies retrieval logic on a small catalog with proxy tasks. It does not measure large-scale index speed or establish APF/online uplift.
Illustrative example: Scanning thousands of items differs from billion-item service.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
How do multitask instructions differ from next-item-only training?
SIGMA: A Semantic-Grounded Instruction-Driven Generative Multi-Task Recommender at AliExpress
Show answer and example
Answer: Instructions specify query, long-tail, discovery, and other goals. Evidence of following them requires task data and evaluation; prompting alone does not teach seasonality.
Illustrative example: The same history can request novelty or a familiar category.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Why combine a short SID prefix with unique item identity?
SIGMA: A Semantic-Grounded Instruction-Driven Generative Multi-Task Recommender at AliExpress
Show answer and example
Answer: The prefix narrows search; unique item representations retain exact identity inside its subcatalog. It is a middle ground between long SID generation and full-catalog ANN.
Illustrative example: Choose a sneaker subcatalog, then distinguish exact models.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why does GR4AD combine business-value weighting and list optimization rather than only predicting the next ad code?
Generative Recommendation for Large-Scale Advertising
Show answer and example
Answer: Next-token imitation can favor frequent low-value ads. Value weighting changes sample contribution; list objectives model relative business utility. Multi-granularity SIDs and relaxed dependencies address separate representation and compute issues. Check relevance and head concentration rather than reporting only revenue proxies.
Illustrative example: A rare high-value ad may matter more than a frequently shown low-conversion one; frequency imitation alone misses that.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Why align GR4AD RSPO's `q_j∝exp(r_j/τ)` with the candidate probability space?
Generative Recommendation for Large-Scale Advertising
Show answer and example
Answer: Normalize within one context's legal candidates and compute cross-entropy against identical student candidate identities. Temperature controls sharpness; ordering, invalid items, and cross-request groups change the target. Public value proxies are not eCPM, and these losses do not verify full LazyAR or dynamic serving.
Illustrative example: If q column two is a camera and student column two is children's clothes, gradients are wrong even if q sums to one.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
Why are GR4AD VSL and RSPO respectively weighted supervision and list alignment?
Generative Recommendation for Large-Scale Advertising
Show answer and example
Answer: VSL emphasizes high-value examples in code-prediction loss; RSPO reshapes within-list preferences using a reward distribution. They act at different granularities but may jointly concentrate exposure. Report concentration alongside NDCG, particularly when local value is proxied by popularity.
Illustrative example: Upweighting costly items and redistributing list probabilities by reward can both increase head exposure.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Why does GR4AD address commercial SIDs, generation dependencies, and beam budgets under load?
Generative Recommendation for Large-Scale Advertising
Show answer and example
Answer: Ads need relevance, business attributes, fast updates, and bounded latency. UA-SID provides multi-granularity codes, LazyAR relaxes sequential dependencies, value/list training aligns utility, and dynamic beams manage serving cost. Next-code accuracy cannot measure all four.
Illustrative example: An excellent ad is unusable if decoding is too slow, while overly narrow beams can miss it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why escalate acquisition from self or teacher to tool research to an expert instead of always asking the strongest expert?
Show answer and example
Answer: Costs differ. Try cheaper potentially adequate paths first and reserve costly help for persistent uncertainty. Thompson sampling combines historical utility and uncertainty, and memory is verified before writing. The cascade orders value and cost decisions, not a claim that cheap sources are always correct.
Illustrative example: Example: Check memory for common configuration, then docs, then an expert only if ambiguity remains.
Sections in the Chinese source: 背景与主要改动, 核心公式与算法. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
What limits follow from deterministic local experts and knowledge sources?
Show answer and example
Answer: They omit real search failures, hallucination, paid models, and token cost. They verify escalation, validation, writes, and proxy cost accounting only. Receiving preset correct knowledge after low confidence does not reproduce HotpotQA, AIME, or real savings.
Illustrative example: Example: A fixture expert always returns a supplied fact. That tests escalation control flow, not real expert request success or cost.
Sections in the Chinese source: 核心公式与算法, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
How does semantic similarity plus Thompson sampling differ from ranking memory by similarity alone?
Show answer and example
Answer: Similarity favors related content. Beta posterior samples explore uncertain memories that may be useful, then outcomes update the posterior. This is not random insertion of irrelevant material. The uncertainty–relevance balance depends on real feedback rather than confidence alone.
Illustrative example: Example: Two records are equally related, one well tested and one new. Sampling can occasionally test the newer one instead of always selecting the highest old score.
Sections in the Chinese source: 背景与主要改动, 核心公式与算法. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Why does U-Mem treat missing knowledge as an active acquisition problem?
Show answer and example
Answer: Similar memories can omit a necessary fact or procedure. Repeated retrieval can leave the same gap. U-Mem learns to acquire information from self, tools, or experts as needed and considers reuse afterward. New knowledge needs provenance and feedback; an expert label does not guarantee truth.
Illustrative example: Example: Memory contains only an obsolete API call. Find new documentation for missing arguments instead of returning the old fragment repeatedly.
Sections in the Chinese source: 背景与主要改动, 核心公式与算法. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why does HiSAC form hierarchical interest agents and route each query to only a few?
HiSAC: Hierarchical Sparse Activation Compression for Ultra-long Sequence Modeling in Recommenders
Show answer and example
Answer: Full-history attention is expensive and one summary can lose long-tail interests. Hierarchical codes and voting create personalized compressed units; queries read only relevant ones. These are not autonomous LLMs, and routing cannot recover interests lost in compression or assignment.
Illustrative example: For lenses, read the photography agent rather than cooking or cycling; discarded astrophotography evidence remains unavailable.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why must HiSAC explicitly handle empty nodes and repeated votes when aggregating behavior by code?
HiSAC: Hierarchical Sparse Activation Compression for Ultra-long Sequence Modeling in Recommenders
Show answer and example
Answer: The node mean is normalized by total valid weight. An empty node provides no evidence of an interest and must not produce a strong signal. Hierarchical votes and query-selected subsets must preserve identity. Repeated behavior should count as defined, but expanding one event across levels must not turn it into several independent consumption events.
Illustrative example: Illustration: One photography event can populate both a coarse and a fine node without implying two purchases. An empty diving node is not a real interest just because its initial vector happens to look similar.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
What choices does HiSAC's hierarchical interest routing retain compared with a single fixed user vector?
HiSAC: Hierarchical Sparse Activation Compression for Ultra-long Sequence Modeling in Recommenders
Show answer and example
Answer: Separate nodes preserve interests at different levels of granularity, and a query can use them as needed instead of averaging away multiple modes. A single vector gives a simpler interface but makes individual sub-interests harder to recover. Routing may miss rare, low-weight needs, and quantization can lose detail. The source's small NDCG increase and Hit decrease should be interpreted separately.
Illustrative example: Illustration: A camera query can read the photography node for a user who likes photography and running, while running shoes can read the sports node. Averaging the two interests into one vector may dilute both.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why aren't HiSAC interest agents a group of live language agents?
HiSAC: Hierarchical Sparse Activation Compression for Ultra-long Sequence Modeling in Recommenders
Show answer and example
Answer: Each is a representation aggregated from hierarchical codes and behavior. Quantization, voting, routing, and weighting operate over related nodes without independent LLM calls. The name describes structure, not agent reasoning or serving cost.
Illustrative example: Photography and camping agents can be two vectors; selecting one needs no inter-agent messages or generated text.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Core mechanism
Why add pseudolabels for unexposed candidates in GPL pre-ranking?
Generative Pseudo-Labeling for Pre-Ranking with LLMs
Show answer and example
Answer: Exposed click examples were selected by the old system and omit many early rejects. An LLM supplies additional labels based on user interest anchors, while a lightweight preranker serves. These noisy judgments are not counterfactual truth; control weight and measure bias.
Illustrative example: A relevant new category may have no logged clicks because it was never shown; estimated relevance does not mean users bought or liked it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Implementation details
Why record GPL real and pseudo supervision sources and λ separately?
Generative Pseudo-Labeling for Pre-Ranking with LLMs
Show answer and example
Answer: Labels differ in reliability and coverage, and the joint coefficient controls pseudo-example influence. Keep provenance; use available history only and exclude test targets from anchors. Public proxies do not reproduce the production LLM labeling system.
Illustrative example: Ten thousand weak labels can overwhelm 100 real clicks if equally weighted; source and weight make that auditable.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Method comparison
What assumption does GPL pseudolabeling address compared with labeling all nonexposures negative?
Generative Pseudo-Labeling for Pre-Ranking with LLMs
Show answer and example
Answer: Missing opportunity is not rejection. Weak labels estimate relevance from intent but can transfer LLM bias; weight them and keep real labels. Removing the LLM from serving does not prove label quality.
Illustrative example: An unseen waterproof shoe can get a weak positive from an interest anchor, yet persuasive ad copy can still fool the teacher.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Basics
Why does GPL target unexposed candidates rather than only adding more exposed-click logs?
Generative Pseudo-Labeling for Pre-Ranking with LLMs
Show answer and example
Answer: The preranker sees a wider pool than final exposure. Interest anchors and LLM weak labels broaden supervision toward retrieval distribution. Inferred labels cannot replace all observed feedback.
Illustrative example: If ten out of 1,000 retrieved items are displayed, the remaining items are not necessarily bad; weak relevance labels do not establish positive user feedback.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Core mechanism
Why separate train/validation/test without student weight updates?
Prompt-Level Distillation: A Non-Parametric Alternative to Model Fine-Tuning for Efficient Reasoning
Show answer and example
Answer: Labeled examples shape rules through extraction, clustering and revision. They carry learned information. Validate and freeze, then never patch from test answers. PLD compiles reasoning into instructions, not parameters.
Illustrative example: Freeze date normalization after validation; adding a test-gold exception leaks even without gradients.
Sections in the Chinese source: 背景与主要改动, 核心公式与数据协议. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Implementation details
When freeze PLD rules, and can test feedback revise them?
Prompt-Level Distillation: A Non-Parametric Alternative to Model Fine-Tuning for Efficient Reasoning
Show answer and example
Answer: Extract/repair on train, use validation for convergence and freeze before test. Prompt changes can leak without weight updates; track versions/provenance.
Illustrative example: Adding a test-specific date exception and rescoring is not independent evaluation.
Sections in the Chinese source: 背景与主要改动, 核心公式与数据协议. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Method comparison
How does semantic DBSCAN differ from string deduplication?
Prompt-Level Distillation: A Non-Parametric Alternative to Model Fine-Tuning for Efficient Reasoning
Show answer and example
Answer: Cluster paraphrased micro-rules and filter noise rather than only exact repeats. Conflicts still require training-error revisions; lists don't ensure consistency.
Illustrative example: Differently worded unit-conversion rules can share a cluster.
Sections in the Chinese source: 背景与主要改动, 核心公式与数据协议. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Basics
Where does PLD store knowledge without parameter fine-tuning?
Prompt-Level Distillation: A Non-Parametric Alternative to Model Fine-Tuning for Efficient Reasoning
Show answer and example
Answer: The teacher abstracts, clusters and revises readable system rules. A student uses frozen instructions at inference. Rules are learned artifacts despite unchanged weights.
Illustrative example: Package date-processing lessons as system instructions.
Sections in the Chinese source: 背景与主要改动, 核心公式与数据协议. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why combine local windows, landmarks, and increasing layer reach?
Bending the Scaling Law Curve in Large-Scale Recommendation Systems
Show answer and example
Answer: Fixed local masks lose remote paths and full attention is expensive. Sparse landmarks provide global connections, wider deeper windows integrate more context, and mixture/system changes support scale. Efficiency depends on windows, landmark counts, and kernels.
Illustrative example: Shallow layers read recent clicks, while landmarks and deeper layers connect older interests without recovering every event.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Can a 48-event ablation prove the 16K production scaling law?
Bending the Scaling Law Curve in Large-Scale Recommendation Systems
Show answer and example
Answer: No. It lacks 18-layer production models, dynamic topology, FP8 kernels, and private logs. Scaling needs matched runs across lengths and hardware measurement.
Illustrative example: A max-length setting of 16000 does not test long history if only dozens are loaded.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How do semi-local attention and deeper wider windows differ from one fixed window?
Bending the Scaling Law Curve in Large-Scale Recommendation Systems
Show answer and example
Answer: Deeper layers gain wider direct access, while fixed windows retain identical local connections per layer. Effective reach also depends on stacking and landmarks; one formula does not cover mixture, precision, and systems changes.
Illustrative example: Read a visit closely in shallow layers, then relate cross-visit interests deeper.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why add landmarks beyond the local ULTRA-HSTU window?
Bending the Scaling Law Curve in Large-Scale Recommendation Systems
Show answer and example
Answer: Local attention preserves detail; sparse landmarks connect distant behavior. LBSL widens deeper layers. Sparse access is not lossless full-HSTU equivalence.
Illustrative example: Read recent events individually and use representative remote positions.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why correct position bias before including session and cross-day creator value in LTV video ranking?
A Long-term Value Prediction Framework In Video Ranking
Show answer and example
Answer: Top positions generate more views, so raw engagement can reward exposure rather than long-term contribution. Correction aims for comparable feedback before session attribution and creator-cycle objectives assign longer-horizon value. Quantile mappings and attribution have assumptions; their combined score is not identified causal retention lift.
Illustrative example: Illustration: A creator always shown first gets more clicks than one shown lower. Align exposure opportunity before judging who drives sustained consumption.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Why must PDQ distinguish the original position distribution from the reference distribution?
A Long-term Value Prediction Framework In Video Ranking
Show answer and example
Answer: Find the outcome's quantile in its original position's conditional distribution, then invert it in a common reference distribution. Inverting in the original distribution would nearly recover the original value. Fit distributions from allowed data and do not tune on future test behavior; without real position logs this is a proxy.
Illustrative example: Illustration: Map a fifth-position video's top-decile watch time to the reference position's top-decile value, not back to the fifth position.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
How does long-term creator value differ from ranking by seconds watched on one video?
A Long-term Value Prediction Framework In Video Ranking
Show answer and example
Answer: Watch time rewards immediate consumption; a cross-day creator objective may reward consistent supply rather than one attention-grabbing video. Define how to combine the two and check whether the long-term proxy favors incumbent creators. The public proxy lacks real creator cycles and does not establish retention or ecosystem gains.
Illustrative example: Illustration: A sensational video can hold attention once, while a reliable series brings people back for days. Watch time alone can miss the latter contribution.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why does LTV Video Ranking correct position before assigning long-term session value?
A Long-term Value Prediction Framework In Video Ranking
Show answer and example
Answer: Position affects views and interactions. PDQ aligns outcomes across positions before aggregating session and cross-day creator value to avoid confusing exposure with contribution. Quantile correction relies on distributional assumptions and does not remove all exposure confounding.
Illustrative example: Illustration: The first video is easier to see than the fifth. Match position conditions before attributing subsequent consumption to creator quality.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Core mechanism
Why doesn't per-input gate softmax necessarily normalize a generative mixture?
A Theoretical Framework for Modular Learning of Robust Generative Models
Show answer and example
Answer: Input-dependent weights cannot leave the global sum like fixed mixture weights. Local unit sums do not guarantee global mass one. Enforce total mass and distinguish worst-domain KL and NLL; silent post-normalization changes the objective.
Illustrative example: A gate can choose the highest-density expert separately at A and B, making their total mass exceed one.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Implementation details
Why can't cached noncausal teacher targets enter the router input?
A Theoretical Framework for Modular Learning of Robust Generative Models
Show answer and example
Answer: Use them only as training targets. Causal router logits depend on prefix alone; the API does not prove the caller's network is causal.
Illustrative example: Knowing the target's expert probabilities during training does not let inference inspect the target token.
Sections in the Chinese source: 核心公式, 本地复现. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Method comparison
Are domain KL and NLL equivalent for gate and adversary updates?
A Theoretical Framework for Modular Learning of Robust Generative Models
Show answer and example
Answer: Within one domain they differ by gate-independent entropy, so gate gradients agree. Across unequal entropies the adversary's ordering can differ. Local practice uses NLL.
Illustrative example: Subtracting different domain entropies can change which domain is worst.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Basics
Why optimize worst-domain performance instead of average alone?
A Theoretical Framework for Modular Learning of Robust Generative Models
Show answer and example
Answer: A good mean can hide severe regressions. Adversarial domain weights adjust input-dependent gates while respecting global probability mass.
Illustrative example: Two strong domains can conceal one failing domain in the average.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Core mechanism
Why learn the index in MFLI and allocate retrieval budget per request and facet?
Rethinking ANN-based Retrieval: Multifaceted Learnable Index for Large-scale Recommendation System
Show answer and example
Answer: One geometry struggles to represent semantics, popularity, and freshness, and an offline index can lag the model. Learned facets preserve distinct structures, and each request allocates candidate quotas among them. Maintain coverage and update consistency; removing traditional ANN does not automatically improve quality and cost.
Illustrative example: Illustration: Exploration can favor freshness, while an exact category can favor semantics. Equal quotas on every request do not exploit context.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Implementation details
Why verify MFLI facet softmax and actual candidate quotas separately?
Rethinking ANN-based Retrieval: Multifaceted Learnable Index for Large-scale Recommendation System
Show answer and example
Answer: Weights summing to one do not establish correct integer allocation. Rounding, total budget, and empty facets matter. Map code identities back to items and deduplicate. The four-facet local proxy does not measure industrial refresh or throughput.
Illustrative example: Illustration: Rounding three equal shares of a ten-item budget yields only nine; define where the remainder goes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Method comparison
How does MFLI allocate budgets differently from concatenating multiple retrieval outputs?
Rethinking ANN-based Retrieval: Multifaceted Learnable Index for Large-scale Recommendation System
Show answer and example
Answer: It jointly designs learned facets and request weights a(q), which affect resource allocation. Simple concatenation can use fixed quotas and unrelated spaces. Learned facets do not guarantee less popularity bias: local head share increased, so inspect the actual distribution.
Illustrative example: Illustration: Search freshness more for new products and semantics more for a specific model. Ten from each branch for every request can duplicate popular items.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Basics
Why learn a retrieval index in MFLI rather than build ANN on existing vectors?
Rethinking ANN-based Retrieval: Multifaceted Learnable Index for Large-scale Recommendation System
Show answer and example
Answer: Independent ANN organizes geometric neighbors; multiple objectives share one space and updates can lag. Learned facets organize semantics, popularity, and time differently and allocate request-dependent budgets. Task linkage does not eliminate update cost or coverage issues.
Illustrative example: Illustration: An item can occupy separate semantic and freshness facets, and the request decides which to search more.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why allow users to correct preferences after action even when prior memory is confident?
Learning Personalized Agents from Human Feedback
Show answer and example
Answer: Confidence describes current memory, not whether preferences changed with time or context. Retrieval and clarification before action address uncertainty; afterward correction replaces stale content. Accepting feedback only at low confidence can permanently lock in learned mistakes or drifted preferences.
Illustrative example: Example: A user previously preferred early flights but changed routines. Update the preference after rejection rather than booking early flights on old confidence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
Why keep current context and remembered preference together in PAHF policy input?
Learning Personalized Agents from Human Feedback
Show answer and example
Answer: Actions depend on instructions, observations, explicit memory, and pre-action feedback. A preference is not a context-free global command. Tie new feedback to its context to avoid converting one situational change into a permanent rule. The local implementation tests memory and feedback contracts, not the original robot or shopping simulator.
Illustrative example: Example: Selecting sugar-free drinks for guests today does not necessarily change personal taste. Keep the scene to understand correction scope.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
How does explicit preference memory differ from providing a hidden user persona as ground truth?
Learning Personalized Agents from Human Feedback
Show answer and example
Answer: Memory comes from actual user feedback and has observable provenance. Hidden persona or gold action belongs only to evaluation. Giving it early removes the personalization problem. PAHF learns preferences and drift through interactions rather than reading a test specification.
Illustrative example: Example: Store that a user said they avoid coffee on workdays. Do not feed the identical statement from a hidden fixture before interaction.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Why do pre-action clarification and post-action correction complement each other?
Learning Personalized Agents from Human Feedback
Show answer and example
Answer: Clarify unknown or uncertain preferences before acting to reduce guesses. Feedback after execution corrects the action and stale memory, handling drift. A confident old belief is not grounds to reject correction when preferences change by context or time.
Illustrative example: Example: Someone formerly liked spicy food but wants mild today. Ask first if uncertain; if an order is already wrong, update based on the explicit correction.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Core mechanism
Why doesn't achieving a simulated shopping goal prove that the simulator behaves like a human?
Show answer and example
Answer: Success measures cooperation with the current recommender. Tone, behavior distributions, and response patterns can still be unrealistic. Population statistics, trained real/fake discrimination, and cross-policy counterfactuals test distinct dimensions. HLS is the classifier's probability of human behavior, not task success or human approval.
Illustrative example: A simulator accepting the first item immediately can succeed quickly without human-like comparison and may fail under another recommender.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Implementation details
Is PLSA a topic model here, and how are local statistical choices documented?
Show answer and example
Answer: No; it is Population-Level Statistical Alignment. Exact permutation for small samples and fixed-seed, plus-one-corrected Monte Carlo for larger ones are explicit local choices. Train and validate the HLS classifier independently rather than filling predictions from gold.
Illustrative example: A passing KS-distance calculation verifies the statistic, not human alignment.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Method comparison
How does cross-recommender validation differ from testing under the policy used to generate simulated data?
Show answer and example
Answer: Same-policy testing may show adaptation to a familiar interaction. Cross-policy tests put a simulator trained under one recommender into another to see whether behavior transfers. This complements distribution matching and discrimination but does not guarantee every deployment policy.
Illustrative example: Train with a popularity-oriented recommender, then check whether shopping responses remain reasonable under an exploration-oriented one.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Basics
Is ConvApparel's PLSA the topic model with the same acronym?
Show answer and example
Answer: No. Population-Level Statistical Alignment compares behavior distributions of simulated and human populations. The local kernel uses KS distance and tie-aware MWU. Group alignment does not establish human realism in every dialogue.
Illustrative example: Comparing human and simulator turn-count distributions is population alignment, not learning latent topics from shopping text.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
How does MixFormer's shared token space change information flow versus concatenating two towers at the end?
MixFormer: Co-Scaling Up Dense and Sequence in Industrial Recommenders
Show answer and example
Answer: Late concatenation finishes field interactions and sequence encoding separately. MixFormer updates both token types together across layers and scales joint modeling. Serving can still decouple reusable user computation from candidate-dependent work rather than rerunning all user history per candidate.
Illustrative example: Illustration: A rainy setting can shape early interpretation of outdoor browsing and reinforce waterproof needs across layers, while late fusion sees two completed summaries.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why check whether candidates enter the shared backbone before reusing one MixFormer user encoding for all dot products?
MixFormer: Co-Scaling Up Dense and Sequence in Industrial Recommenders
Show answer and example
Answer: If a candidate modifies the unified user state, swapping it invalidates reuse. The local model jointly encodes user fields and history first, then scores separate candidate vectors. Verify actual dependencies, not naming. Public genres replacing private fields also limit fidelity.
Illustrative example: Illustration: Encode four user fields and 32 events once and dot-product with shoes or a camera. If the camera went into the backbone, reusing that state for shoes would change the path.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
Why does a parameter-matched stacked baseline isolate MixFormer's architecture better than the old fixed semantic gate?
MixFormer: Co-Scaling Up Dense and Sequence in Industrial Recommenders
Show answer and example
Answer: Actually train stacked and unified models with similar budgets to compare interaction structure. A fixed score gate only proxies fusion. Matching parameters does not remove optimization, feature, or variance differences. The source retired the old proxy; do not mix its numbers with the new experiment or claim stable industrial gains.
Illustrative example: Illustration: Comparing two trained architectures is meaningful; adding a semantic constant to existing scores tests a rule, not unified Transformer training.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why include both dense fields and behaviors in MixFormer's unified token space?
MixFormer: Co-Scaling Up Dense and Sequence in Industrial Recommenders
Show answer and example
Answer: Separate encoders postpone communication. Joint layers update both representations and scale their capacity together. Heterogeneous fields need proper projection, not treatment as literal words. Serving still separates user and candidate dependencies; not every state can be shared.
Illustrative example: Illustration: A rain field helps interpret outdoor clicks early, and the refined interest feeds back into field interactions before any final concatenation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
How does Climber-Pilot account for farther-future consumption without serving a full rollout?
Climber-Pilot: A Non-Myopic Generative Recommendation Model Towards Better Instruction-Following
Show answer and example
Answer: Time-aware multi-item prediction trains the current state on several future targets, embedding longer-horizon structure in the parameters. Serving can still generate one result; condition-guided attention adds category and business constraints without extra rollout steps. True future items belong only to training supervision.
Illustrative example: Training can learn that camera interest extends to accessories. Serving cannot read a user's future purchase to recommend now.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
How should TAMIP time weights interact with valid-target masks in the multi-future loss?
Climber-Pilot: A Non-Myopic Generative Recommendation Model Towards Better Instruction-Following
Show answer and example
Answer: Offset k needs a real target and its specified weight `w_k`; padding at the tail of short sequences is not positive supervision. Offsets can share a current state whose inputs stop at the cutoff. Time-weighted losses and conditional attention masks affect different tensors.
Illustrative example: With two future events, compute two loss terms. A third empty slot creates no fake target, and heavier weighting does not let a future event enter history.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
At what stage do constraints act in Climber-Pilot's conditional sparse attention versus post-decoding filtering?
Climber-Pilot: A Non-Myopic Generative Recommendation Model Towards Better Instruction-Following
Show answer and example
Answer: An internal attention mask changes which evidence forms the representation; post-filtering only removes generated results and cannot recover legal options that generation omitted. Internal masks must match business rules and do not eliminate the need for legality checks.
Illustrative example: A category mask can guide candidate formation. Removing all invalid candidates after generation can leave an empty list and cannot recover spent generation budget.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Why can Climber-Pilot predict multiple future items in training but serve in one step?
Climber-Pilot: A Non-Myopic Generative Recommendation Model Towards Better Instruction-Following
Show answer and example
Answer: Offset targets inject long-horizon consumption structure into a shared current state, and time weights differentiate offsets. Serving uses that learned model directly without simulating the future or seeing those targets.
Illustrative example: Train the current state to predict the upcoming camera, lens, and bag; score today's candidates without access to those true future events.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Core mechanism
Why separate baseline and coupon sensitivity before comparing predictions?
Jointly Optimizing Debiased CTR and Uplift for Coupons Marketing: A Unified Causal Framework
Show answer and example
Answer: Natural propensity can masquerade as coupon gain. Two towers model baseline/treatment; counterfactual constraints link unobserved cases, and unit uplift normalizes tier increments. Identification still depends on assignment and assumptions.
Illustrative example: A naturally 80%-likely buyer may gain less than one moving 20%→50%, subject to selection bias.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Implementation details
Does local UniMVT have randomized coupon tiers and counterfactuals?
Jointly Optimizing Debiased CTR and Uplift for Coupons Marketing: A Unified Causal Framework
Show answer and example
Answer: No. MovieLens checks two-tower and counterfactual constraints only. Causal estimates require overlap and confounding control; test outcomes must not define treatment.
Illustrative example: Assigning coupons by rating leaks outcomes into group assignment.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Method comparison
Why isn't treatment CTR uplift?
Jointly Optimizing Debiased CTR and Uplift for Coupons Marketing: A Unified Causal Framework
Show answer and example
Answer: Uplift compares treated and untreated potential outcomes under matched conditions; high treated CTR can reflect selection. Counterfactual identification needs assumptions, not just two predictions.
Illustrative example: Coupons targeted to high-intent users can show high CTR without causing it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Basics
Why separate baseline click propensity and coupon-strength sensitivity?
Jointly Optimizing Debiased CTR and Uplift for Coupons Marketing: A Unified Causal Framework
Show answer and example
Answer: Users may buy anyway. Separate natural propensity from treatment response to estimate increment rather than conversion under a coupon.
Illustrative example: A coupon used by someone who would buy anyway does not necessarily generate a sale.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
What is the trade-off between Gaussian soft assignment and hard nearest-center quantization?
Show answer and example
Answer: Hard boundaries can jump and leave centers unused. Soft posteriors combine means and pass a smoother residual to the next stage, but must connect to discrete serving. Decoupling from CTR does not guarantee click-optimal codes.
Illustrative example: A cooking/lifestyle clip can combine both categories before the next residual stage.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
What should the next RQ-GMM residual subtract?
Show answer and example
Answer: Subtract the current posterior-weighted reconstruction, not a hard maximum center. Normalize probabilities and record codebook versions separately; local features proxy production multimodal inputs.
Illustrative example: A hard subtraction after soft reconstruction changes what the next layer learns.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
How does separate semantic coding differ from letting CTR update the codebook?
Show answer and example
Answer: Decoupling preserves general semantics without guaranteeing CTR optimality. End-to-end clicks can reshape code assignments and alter stability; this is a coupling comparison, not a measured ranking.
Illustrative example: Similar items can start with similar codes, but CTR gradients might redistribute codes by popularity.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why quantize residuals repeatedly instead of clustering once?
Show answer and example
Answer: Early stages encode coarse semantics and later ones encode unexplained details. Soft Gaussian posterior assignments can improve utilization, depending on clusters and features.
Illustrative example: Encode category first, then details instead of asking one code to represent everything.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why does RoleGen infer conversion-path functions rather than rely on similarity to reactivate dormant users?
Awakening Dormant Users: Generative Recommendation with Counterfactual Functional Role Reasoning
Show answer and example
Answer: For inactive users, repeatedly serving similar old items can narrow interests. Functional roles model exploration, comparison, and conversion. Counterfactual paths estimate an item's instrumental contribution, which feeds the SID generator and iterative reflection. These estimates rely on model and data assumptions, not causality identified by observed trajectories alone.
Illustrative example: A camera buyer may have watched reviews and photography tutorials. A tutorial is not similar to a camera listing but may reactivate interest.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Can RoleGen's local public data establish production conversion among dormant users?
Awakening Dormant Users: Generative Recommendation with Counterfactual Functional Role Reasoning
Show answer and example
Answer: No. A MovieLens subset and proxy objective check the core path, without private dormant-user logs or actual traffic buckets. Log score provenance, candidates, and feedback. Do not present proxy deltas as live intervention effects.
Illustrative example: A movie preference ranking change is not causal lift in e-commerce conversion.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
How does counterfactual instrumental contribution differ from co-occurrence similarity?
Awakening Dormant Users: Generative Recommendation with Counterfactual Functional Role Reasoning
Show answer and example
Answer: Co-occurrence describes what appears together. Instrumentality asks how an outcome might change if the item were removed. A local proxy and do notation do not identify real causal effects.
Illustrative example: Phones and chargers may be bought together without charger recommendations causing phone purchases.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Does “role” in RoleGen mean a user identity or an item's function along a conversion path?
Awakening Dormant Users: Generative Recommendation with Counterfactual Functional Role Reasoning
Show answer and example
Answer: It means an item's function in the conversion path, not a user occupation or demographic. An LLM Reasoner infers item roles and passes candidates to the SID generator and feedback loop. The roles are model interpretations, not observed causal identities.
Illustrative example: An item can help comparison in one trajectory and be the purchase in another, regardless of whether the user is a student.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why does RGAlign use the ranker's item preferences to choose query winners and losers?
RGAlign-Rec: Ranking-Guided Alignment for Latent Query Reasoning in Recommendation Systems
Show answer and example
Answer: A fluent latent query can miss items the ranker values. Generate alternatives, rank the items they retrieve, and derive demonstrations and preference pairs to align query generation with top-ranked retrieval. RG-SFT learns the mapping and DPO strengthens relative preferences. Supervision still inherits the ranker's judgment, not independent satisfaction.
Illustrative example: History may suggest beginner photography or professional lenses. If ranking favors a lightweight beginner camera, prefer the first query rather than the one phrased most naturally.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
How should we distinguish local RGAlign queries from the production LLM pipeline?
RGAlign-Rec: Ranking-Guided Alignment for Latent Query Reasoning in Recommendation Systems
Show answer and example
Answer: A fixed public MovieLens candidate protocol checks the ranking-guided core, not private features, the production base, or traffic. Compare positive and negative queries under the same context and candidates. Do not choose winners with test labels.
Illustrative example: Better offline candidate scores do not establish higher live conversion.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
How does ranking-guided query alignment differ from ordinary language SFT?
RGAlign-Rec: Ranking-Guided Alignment for Latent Query Reasoning in Recommendation Systems
Show answer and example
Answer: SFT maximizes demonstration text likelihood. Ranking guidance constructs preferred and rejected queries from downstream item preferences, tying verbalized intent to ranking. Ranker preferences can be biased and are not causal user ground truth.
Illustrative example: A fancier query may retrieve worse candidates; downstream feedback can distinguish the two.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Is an RGAlign-Rec latent query an actual search term typed by the user?
RGAlign-Rec: Ranking-Guided Alignment for Latent Query Reasoning in Recommendation Systems
Show answer and example
Answer: No. It is verbalized intent generated from user context to connect item retrieval with ranking needs. Fluent text does not establish that the user said it. Distinguish observed behavior from inferred intent to understand the supervision.
Illustrative example: After a user views camping content, the model may infer “lightweight tent” even if no such search appears in the log.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Core mechanism
Why does CAPTS value a trigger through downstream content consumption instead of trigger clicks?
CAPTS: Channel-Aware, Preference-Aligned Trigger Selection for Multi-Channel Item-to-Item Retrieval
Show answer and example
Answer: A trigger is an entry point, and its direct feedback may not describe what it retrieves. VAM attributes consumption of two-hop content to the trigger; CATR jointly assigns triggers to I2I channels while rewarding value and penalizing overlap. Popular entry points can waste limited candidate slots by returning the same content.
Illustrative example: Two popular triggers retrieve the same comedy videos, while a less popular one adds valuable science videos. Value and complementarity can guide a better allocation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Implementation details
Why retain trigger, channel, and downstream item indexes when aggregating CAPTS value?
CAPTS: Channel-Aware, Preference-Aligned Trigger Selection for Multi-Channel Item-to-Item Retrieval
Show answer and example
Answer: The same trigger can return different candidates and contributions by channel. Attribute weights and feedback to the correct path and use it to compute channel overlap during joint selection. A global item score loses trigger credit. Public proxy consumption labels are not production value.
Illustrative example: When two triggers retrieve the same item, do not credit each with the entire consumption event. Keep path and weight to identify what each trigger actually added.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Method comparison
How does CAPTS joint channel routing differ from selecting the strongest trigger for each I2I channel independently?
CAPTS: Channel-Aware, Preference-Aligned Trigger Selection for Multi-Channel Item-to-Item Retrieval
Show answer and example
Answer: Independent optimization can send every channel through the same trigger and produce overlapping sets. The joint objective balances downstream value against overlap to allocate distinct roles. Coverage may trade against precision; the reported task measurements are not proof of cost-free improvement across every channel.
Illustrative example: Rather than retrieving the same popular sequels three times, let one channel keep them and another explore related work through director interest.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Basics
Why define CAPTS trigger value after the two-hop retrieval chain?
CAPTS: Channel-Aware, Preference-Aligned Trigger Selection for Multi-Channel Item-to-Item Retrieval
Show answer and example
Answer: A trigger retrieves other content; its clicks need not predict the value of those candidates. VAM attributes downstream consumption back to the entry point within its I2I channel; CATR allocates routes for complementary coverage. Trigger value depends on the downstream channel.
Illustrative example: A popular watched video may yield redundant candidates, while an obscure history item surfaces a new topic the user likes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why check conflicts against the entire original sample?
Online Advertising with Spatial Interactions
Show answer and example
Answer: Rounding probabilities derive from independent sampling under geometric and group constraints. Greedy acceptance changes conditional distributions. Local tests verify actual LP/rounding, not advertising uplift.
Illustrative example: Accepting A first would change B's decision; the paper instead evaluates both against one fixed sample.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
How handle disk boundaries and invalid probabilities?
Online Advertising with Spatial Interactions
Show answer and example
Answer: Cover only at strictly smaller distance than radius. Validate metric and thinning probabilities; fail rather than clip. Local HiGHS LP does not reproduce all auctions or the 2D PTAS.
Illustrative example: Probability above one cannot be silently clipped without changing the distribution.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
How does LP with two-stage rounding differ from greedily placing ads?
Online Advertising with Spatial Interactions
Show answer and example
Answer: Solve the fractional program, test conflicts on the entire first-stage sample, then thin again. Greedy tests only already-kept items and changes probabilities and guarantees.
Illustrative example: A sampled item discarded later can still create a first-stage conflict.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why model interference between ad positions?
Online Advertising with Spatial Interactions
Show answer and example
Answer: Neighboring ads affect value. Grouped disks and coverage constraints represent compatible selections, with LP and randomized rounding.
Illustrative example: Two nearby large ads can compete for attention, so values are not independent.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why compress before every explicit cross in MLCC instead of compressing only the initial input?
Show answer and example
Answer: Layer states contain different interaction orders. Recompressing the current state lets the next cross use newly formed information. Dynamic crosses in compact space project back into the backbone to reduce high-dimensional cost. Parallel subspaces can cover different relations but need to retain useful directions.
Illustrative example: Illustration: First combine region and category, then compress that combination to cross it with price preference. Reusing only initial fields misses the new relationship.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why must MLCC's Cl and Pl match the corresponding layer state?
Show answer and example
Answer: Compress current xl to z, compute its dynamic gate from z, and project back to xl's width. Reusing the initial compression or the wrong layer's weights weakens progressive high-order updates. Check that each layer actually receives the updated state, not just valid residual shapes.
Illustrative example: Illustration: Layer two should compress a state that includes layer one's crosses; compressing x0 again can just repeat the first cross.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How do MC-MLCC's parallel subspaces differ from widening one compression space?
Show answer and example
Answer: Separate subspaces can specialize in distinct crosses and combine outputs; one wider projection concentrates capacity. Parallel channels can specialize or duplicate each other. Match parameters and FLOPs; the mechanism does not establish universal cost and accuracy gains.
Illustrative example: Illustration: Price–budget and time–context can occupy different channels, but if both learn popularity, capacity is redundant.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why project MLCC's compact-space crosses back into a high-dimensional backbone?
Show answer and example
Answer: Compact z saves interaction cost, while the residual backbone retains old information. Projection P injects new combinations rather than trapping all layers in one reduced space. Compression may omit directions, and residuals do not guarantee learning every high-order cross.
Illustrative example: Illustration: Compute budget–category interaction in a small space and add it back, leaving other fields available at the next layer.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why does LinkedIn Feed's sequential model stress evaluation without random negative sampling?
An Industrial-Scale Sequential Recommender for LinkedIn Feed Ranking
Show answer and example
Answer: A few easy random negatives simplify ranking and conceal hard distinctions in a large pool. A model that jointly processes candidates, history, and context needs a consistent candidate protocol to test scaling and incremental serving. A local proxy is not LinkedIn's production distribution, and gains cannot be compared across different negative counts.
Illustrative example: Illustration: Finding one positive among nine obviously irrelevant items differs from finding it among thousands of similar Feed candidates, even with the same model.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why validate candidate-independent history state before incremental serving in LinkedIn Feed?
An Industrial-Scale Sequential Recommender for LinkedIn Feed Ranking
Show answer and example
Answer: A reusable prefix must not depend on the candidate being swapped. History and model changes also invalidate caches. Candidate attention may read shared state but must not reuse candidate-conditioned history for other candidates. The local core proxy lacks the full LinkedIn system and does not measure true long-sequence throughput.
Illustrative example: Illustration: A new article may read old history K/V through a different query. If the old article was written into that state, sharing it with the new one leaks candidate information despite fast execution.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
Why is evaluation without random negatives harder than testing one positive and a few random negatives?
An Industrial-Scale Sequential Recommender for LinkedIn Feed Ranking
Show answer and example
Answer: Sampled sets often omit real competitors. Metrics vary with negative count and difficulty, potentially overstating ranking ability. A fuller pool better represents serving competition but costs more. Evaluation without random negatives does not imply no training negatives or justify comparing percentages across protocols.
Illustrative example: Illustration: Ranking among 99 irrelevant articles is easier than beating many similar articles in the full professional corpus. The metric can be named NDCG in both without measuring the same task.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why does LinkedIn Feed combine candidates, interaction history, and context into a sequence?
An Industrial-Scale Sequential Recommender for LinkedIn Feed Ranking
Show answer and example
Answer: They jointly determine current content value, so a common backbone lets evidence interact before the final stage. Very long history retains career interests but adds compute and candidate dependence, motivating incremental serving. A unified representation does not make all states reusable across candidates.
Illustrative example: Illustration: Recent job seeking, a long-running interest in engineering, and a current job-listing candidate together explain value better than a set of liked articles.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Core mechanism
Why roll out without experience and score with an experience-aware teacher?
Show answer and example
Answer: Internalize external knowledge rather than preserve dependency. Student gets task only; teacher scores same prefix with experience; update by student-to-teacher reverse KL. Reliability matters and local checks don't reproduce long system prompts.
Illustrative example: Remove the instruction at evaluation to test whether the operation lesson lives in weights.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Implementation details
What is OPCD reverse-KL direction and student input?
Show answer and example
Answer: KL from no-experience student to experience-conditioned teacher on identical student prefix. Experience stays teacher-only. Local conditional distributions aren't long prompts or paper environment.
Illustrative example: Swapping numerator to teacher changes KL direction.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Method comparison
How does OPCD differ from retrieving the same experience at inference?
Show answer and example
Answer: Retrieval depends on external context and read cost. Distillation reduces that dependence but can forget or overgeneralize. They can complement each other; compare empirically.
Illustrative example: Updated documents can be swapped immediately; older knowledge distilled into weights doesn't automatically update.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Basics
What context-dependent learning can OPCD preserve?
Show answer and example
Answer: Prompt, retrieval or past experience vanishes when context clears. Distill a context-rich teacher into a context-free student on its own trajectories to internalize behavior.
Illustrative example: Learn a tool-use lesson that remains useful when the hint is absent next time.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Core mechanism
Why block candidate-to-user flow and then compensate?
Compute Only Once: UG-Separation for Efficient Large Recommendation Models
Show answer and example
Answer: Isolation makes user encoding reusable; it may weaken bidirectional expressiveness, so compensation restores interactions in the item scoring path only. Do not cache compensated state as public.
Illustrative example: Compute one item-free user state for 100 candidates, then apply separate per-item compensation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Implementation details
How test whether the UG-Sep mask permits safe sharing?
Compute Only Once: UG-Separation for Efficient Large Recommendation Models
Show answer and example
Answer: Fix user and vary candidates: protected user representations must stay unchanged while item output may change. Check attention and TokenMixer, not only masks. No production kernels/QPS.
Illustrative example: A masked attention path is insufficient if the mixer leaks candidate information back.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Method comparison
Can Information Compensation turn candidate-conditioned state into reusable cache?
Compute Only Once: UG-Separation for Efficient Large Recommendation Models
Show answer and example
Answer: No. It restores candidate-specific scoring interactions. Shared state must remain candidate-independent, so caching compensation for another item reintroduces contamination.
Illustrative example: Item A's compensated state is not general user state for B.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Basics
Do UG-Sep user and candidate tokens describe two users?
Compute Only Once: UG-Separation for Efficient Large Recommendation Models
Show answer and example
Answer: No. One request has a shared user block and an item-specific candidate block. Isolation controls information flow, not user groups.
Illustrative example: Several candidate blocks can reuse the same user's history.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why add bounded semantic advantage only when it agrees with business advantage?
S-GRec: Personalized Semantic-Aware Generative Recommendation with Asymmetric Advantage
Show answer and example
Answer: The rewards have different scales. A2PO standardizes each within rollout groups, anchors to business advantage, and adds semantic signal only with matching sign and bounded magnitude. This does not remove reward bias.
Illustrative example: A pleasing rationale cannot turn negative business feedback positive.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Is local S-GRec business reward real eCPM?
S-GRec: Personalized Semantic-Aware Generative Recommendation with Asymmetric Advantage
Show answer and example
Answer: No. Public data lacks money signals; it uses next-item ranking. Judge, generation, and policy run at reduced scale with different labels.
Illustrative example: A correct next movie is behavioral evidence, not ad revenue.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
How do user-conditioned aspect weights differ from one fixed vector?
S-GRec: Personalized Semantic-Aware Generative Recommendation with Asymmetric Advantage
Show answer and example
Answer: The judge supplies aspect scores and an aggregator gives normalized nonnegative per-user weights. Fixed weights express one trade-off; conditional weights personalize the same content but do not prove satisfaction.
Illustrative example: One user prioritizes history fit while another prioritizes novelty.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Why keep the large judge off S-GRec's online generation path?
S-GRec: Personalized Semantic-Aware Generative Recommendation with Asymmetric Advantage
Show answer and example
Answer: Offline judging supplies per-aspect preferences; a lightweight generator emits SIDs without online LLM latency. Judge quality still depends on training data and is not cost-free.
Illustrative example: An expensive teacher trains semantic preference while a small model serves.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why does CADET put cacheable history and post-scoring context in different information-flow stages?
CADET: Context-Conditioned Ads CTR Prediction With a Decoder-Only Transformer
Show answer and example
Answer: The cache must remain valid when candidates or later context change. CADET encodes history and scores candidates before injecting post-scoring context at the allowed stage, preventing use of information unavailable at serving time. Mixing everything from the start can invalidate reuse and create training-serving mismatch despite better offline scores.
Illustrative example: Do not write ad A's post-scoring context into shared history before evaluating ad B. Keep the shared prefix distinct from later conditional state.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
When does new CADET context invalidate cached history K/V?
CADET: Context-Conditioned Ads CTR Prediction With a Decoder-Only Transformer
Show answer and example
Answer: Check whether it changes history inputs, masks, or model parameters. The shared K/V stays valid only if context enters a later candidate-query path. The source formula lets a context query read prior history; it does not authorize feeding arbitrary context back into the cached backbone. Inspect actual tensor dependencies.
Illustrative example: A change to the query can reuse the history K/V; changing history time encodings requires recomputation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does staged context in CADET differ from feeding all log fields into a Transformer together?
CADET: Context-Conditioned Ads CTR Prediction With a Decoder-Only Transformer
Show answer and example
Answer: Indiscriminate concatenation can expose unavailable fields and leak candidate computations into shared history. Separating cacheable state from post-scoring context limits arbitrary interactions but preserves serving consistency. Local public-data proxies do not verify the full timing of production advertising features.
Illustrative example: Mixing current-request features with the next action in one sequence invites a shortcut; stage boundaries allow only evidence available when the request arrives.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why does CADET emphasize when post-scoring context becomes visible?
CADET: Context-Conditioned Ads CTR Prediction With a Decoder-Only Transformer
Show answer and example
Answer: Training logs may include information created only after scoring. Using it before scoring misaligns training and serving. Encode cacheable history first, then introduce context only at its legitimate stage; timing is part of the feature contract.
Illustrative example: A user's reaction after an impression can enter the next history, but cannot predict the current click before that impression.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Core mechanism
Why combine fixed shared activation, adaptive private experts, and global balance?
SMES: Towards Scalable Multi-Task Recommendation via Expert Sparsity
Show answer and example
Answer: Shared capacity handles commonality, private routing handles task differences, and global balance detects concentration hidden within tasks. Actual specialization and balance penalties still require evaluation.
Illustrative example: Clicks and purchases can share interest experts while choosing novelty or price specialists, unless global dispatch overloads one.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Implementation details
Can balancing be computed separately per task only?
SMES: Towards Scalable Multi-Task Recommendation via Expert Sparsity
Show answer and example
Answer: No. Separate balance can miss aggregate expert overload. Use cross-task counts. MovieLens lacks production sparse scheduling, so dispatch counts are not throughput.
Illustrative example: Individually spread tasks can all favor the same two experts globally.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Method comparison
How do shared/private pools differ from one MoE router?
SMES: Towards Scalable Multi-Task Recommendation via Expert Sparsity
Show answer and example
Answer: Separate pools constrain common and task-specific selections; one pool lets all compete. This does not guarantee less negative transfer.
Illustrative example: Reserving all shared capacity for purchases can harm other tasks.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Basics
Why use both shared and private experts in SMES?
SMES: Towards Scalable Multi-Task Recommendation via Expert Sparsity
Show answer and example
Answer: Stable shared experts hold common signals, task-private experts preserve differences, and global load balances cross-task dispatch.
Illustrative example: Viewing and buying share interest features but can retain distinct decision experts.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Core mechanism
Why should the outer state machine control real A/B promotion?
Self-Evolving Recommendation System: End-To-End Autonomous Model Optimization With LLM Agents
Show answer and example
Answer: Offline agents cheaply screen ideas but cannot capture all user and serving constraints. The outer loop controls safety, resources, promotion, and journal feedback. Local loops do not replace live production A/B.
Illustrative example: Reject an offline winner with excess latency; record a later approved experiment so future rounds learn.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Implementation details
When does a local evolve operator count as integrated?
Self-Evolving Recommendation System: End-To-End Autonomous Model Optimization With LLM Agents
Show answer and example
Answer: The controller must execute changes, train and evaluate on isolated validation, and leave traceable state/journal entries. Registration or simulated plans are not execution or real A/B.
Illustrative example: An optimizer suggestion that cannot run remains a proposal.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Method comparison
How does bilevel optimization differ from a hyperparameter sweep?
Self-Evolving Recommendation System: End-To-End Autonomous Model Optimization With LLM Agents
Show answer and example
Answer: Inner training explores architecture, optimizer, or reward; outer governance manages online goals, resources, safety, and accumulated knowledge. Tuning can support the inner layer but does not replace the outer loop.
Illustrative example: A learning-rate sweep does not decide whether to spend live traffic or archive failures.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Basics
Does Self-Evolving Rec's journal store model parameters or experimental knowledge?
Self-Evolving Recommendation System: End-To-End Autonomous Model Optimization With LLM Agents
Show answer and example
Answer: It records proposals, execution, and protocol-bound outcomes. This is distinct from checkpoint storage. A prose plan without runnable candidates and evaluation is not an experiment.
Illustrative example: Record a failed optimizer comparison so the next iteration avoids repeating it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why generate SARM anchors offline and update lightweight ranking online?
SARM: LLM-Augmented Semantic Anchor for End-to-End Live-Streaming Ranking
Show answer and example
Answer: MLLMs enrich live content but are expensive per request. Offline anchors feed lightweight ranking that learns real-time user/item matching. Stale or incomplete anchors still constrain results.
Illustrative example: A ranker can use an equipment-demo anchor, but it needs refresh when the live topic changes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
Does local SARM generate and deploy production live anchors?
SARM: LLM-Augmented Semantic Anchor for End-to-End Live-Streaming Ranking
Show answer and example
Answer: No. Fixed MovieLens candidates test semantic injection and scoring, not private streams, real MLLMs, or clusters. Do not confuse the anchor encoder with a sparse autoencoder by the acronym SAE.
Illustrative example: An auditable vector proxy is not a full livestream summary.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
Do anchors and online ID/behavior features carry the same information?
SARM: LLM-Augmented Semantic Anchor for End-to-End Live-Streaming Ranking
Show answer and example
Answer: No. Anchors describe content; IDs and behavior capture interaction and fit. Neither replaces the other, and asynchronous semantic freshness matters.
Illustrative example: Two camping streams can have similar descriptions but different engagement histories.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Is a SARM semantic anchor a new item ID?
SARM: LLM-Augmented Semantic Anchor for End-to-End Live-Streaming Ranking
Show answer and example
Answer: No. Offline multimodal content understanding produces it for a lightweight ranker. Identity and behavior stay separate; an anchor is not evidence of user interaction.
Illustrative example: A stream keeps its ID while its anchor describes a changing topic.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why interleave sequence and interaction blocks in Kunlun rather than making only one tower larger?
Show answer and example
Answer: The sequence model handles historical dependencies and the heterogeneous interaction model handles field relationships. Scaling one side alone can leave the other as an information bottleneck. Kunlun exchanges sequence seeds and non-sequence information between layers, uses gating and residuals to control deep updates, and relies on parallel engineering to scale. Joint structure does not make every module free to reuse: distinguish candidate dependence from serving cost.
Illustrative example: Illustration: A long history reveals outdoor interest, but price and context interactions arriving only at the end cannot guide deeper interpretation. Interleaving lets them cooperate across layers instead of concatenating once.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why must Kunlun's CompSkip align the dimensions of the old state and F output?
Show answer and example
Answer: h(l+1)=αhl+(1−α)Fl(hl) combines states in the same representation space. Specify the range and broadcast axis of α rather than gating across candidates or batch examples accidentally. Respect synchronized sequence and interaction updates. The four-layer local compact core's quality variation is not production throughput evidence.
Illustrative example: Illustration: A per-layer α must not accidentally become candidate-dependent and contaminate a shared state merely because broadcasting succeeds.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
Where does Kunlun's gated GDPA affect evidence compared with ordinary softmax attention?
Show answer and example
Answer: The gate's log-sigmoid is added to attention logits, suppressing relations before normalization rather than scaling the final vector afterward. It can suppress noise or rare important events. CompSkip separately changes residual depth.
Illustrative example: Illustration: Downweighting an accidental click before normalization reallocates attention to other events; multiplying the whole user vector afterward cannot reproduce that change.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why does Kunlun put a sequence block and an interaction block in the same layer?
Show answer and example
Answer: The sequence block extracts behavioral dependencies and the interaction block combines heterogeneous fields. Global Interaction passes information between layers to prevent isolated towers from hitting a communication bottleneck. HSP summarizes sequence seeds, while system parallelism and CompSkip address deep scaling. Enlarging one FFN does not reproduce these separate mechanisms.
Illustrative example: Illustration: History reveals camping intent, and a budget field in the interaction block changes its interpretation and later fusion. Towers concatenated only at the end see each other's clues much later.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why does DiffuReason iteratively refine latent intent through diffusion instead of predicting one final recommendation representation?
DiffuReason: Bridging Latent Reasoning and Generative Refinement for Sequential Recommendation
Show answer and example
Answer: One prediction can compress conflicting interests into an unstable direction. Latent Thinking Tokens carry intent and iterative denoising revisits cues; GRPO aligns continuous generation with discrete ranking feedback. Extra steps cost compute and can accumulate bias; a local proxy does not establish full latent reasoning capability.
Illustrative example: Fitness and dessert signals can be reconciled for a request without interpreting intermediate vectors as literal human thoughts.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Why is a fixed reward added to final scores insufficient to implement DiffuReason denoising and GRPO?
DiffuReason: Bridging Latent Reasoning and Generative Refinement for Sequential Recommendation
Show answer and example
Answer: Denoising needs defined timesteps, state transitions, and noise. Policy updates need same-context rollouts, rewards, and matching probabilities. A score addition does not train the continuous generation distribution or establish gradients. Local proxies do not replace end-to-end evidence.
Illustrative example: Multiplying popularity by 0.1 changes ranking but not latent denoising. Verify trajectories and real parameter updates before claiming policy training.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
What benefits and serving costs does DiffuReason's iterative update assume versus a one-shot user vector?
DiffuReason: Bridging Latent Reasoning and Generative Refinement for Sequential Recommendation
Show answer and example
Answer: Denoising can revisit history at intermediate states, adding iteration cost and sampling uncertainty. Repeating the same bias need not help. Keep public proxies separate from actual diffusion policy training; a final score cannot verify the full path.
Illustrative example: Later steps might correct an initial popularity bias if niche evidence is available, but ten steps cannot invent evidence missing from conditioning.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Why denoise DiffuReason's continuous Thinking Tokens rather than generating a long language rationale?
DiffuReason: Bridging Latent Reasoning and Generative Refinement for Sequential Recommendation
Show answer and example
Answer: Continuous variables can hold several interest cues and be revised from history without language decoding at each step. Discrete ranking reward aligns the internal process through policy training. Latents are not automatically readable reasons, and iteration does not guarantee conflict resolution.
Illustrative example: A vector may adjust novelty and budget preferences without a textual explanation proving each choice.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why retain multiple SIDs per item and choose by recommendation loss in PIT?
PIT: A Dynamic Personalized Item Tokenizer for End-to-End Generative Recommendation
Show answer and example
Answer: One fixed code constrains all users to one semantic path and may drift out of fit. PIT jointly trains tokenizer and recommender and selects among codes by recommendation loss. A one-to-many index preserves choices but complicates mapping and deduplication; multiple paths are not multiple items.
Illustrative example: Illustration: Office stretches may be reachable via fitness or desk health. Route by user, then merge both paths to the same video.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
Why use PIT's user-guided minimum-loss choice only among training candidate codes?
PIT: A Dynamic Personalized Item Tokenizer for End-to-End Generative Recommendation
Show answer and example
Answer: Argmin can use training labels to select a supervisory path, but inference must use model probabilities and legal beam paths rather than gold loss. Alternate codes map back to the same item. Gold-guided code choice is answer-aware routing.
Illustrative example: Illustration: Knowing the correct video and picking its easiest code inflates Hit. Generate codes before checking the item.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
Where do gradients couple PIT's tokenizer and recommender compared with a frozen codebook?
PIT: A Dynamic Personalized Item Tokenizer for End-to-End Generative Recommendation
Show answer and example
Answer: A frozen generator adapts to existing codes; PIT uses recommendation loss in code selection and learning, adapting paths to preference. This can destabilize addresses, so track versions and drift. Neither lossless streaming updates nor universal inferiority from a local negative proxy is established.
Illustrative example: Illustration: Keep an office-use entry for a fitness video if recommendation supervision supports it; a frozen index can only change probabilities on existing paths.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why can distribution drift make old PIT SID paths inappropriate even when item text is unchanged?
PIT: A Dynamic Personalized Item Tokenizer for End-to-End Generative Recommendation
Show answer and example
Answer: Needs, inventory, and interaction frequency change, weakening the old mapping to the generative distribution. Distinguish actual data drift, model updates, and address changes from random variation. One fluctuation does not prove recoding is needed.
Illustrative example: Illustration: The same light coat serves spring commuting and autumn travel; a fixed random seed cannot remove changing preferences.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why apply both low-rank projection and a learned mask before ML-DCN's explicit crosses?
Show answer and example
Answer: Full-width crosses increase compute with capacity. Low rank maps to a configurable small dimension and back; the mask selects participating channels. These control cost and information separately. Residuals preserve the original state, but a low-rank bottleneck can still miss relationships outside its subspace.
Illustrative example: Illustration: Rather than fully crossing thousands of age, city, and device features, select useful channels and combine them in tens of dimensions, leaving budget for depth.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
What shape relationships should ML-DCN's V, U, mask, and x0 satisfy?
Show answer and example
Answer: Mask the current xl channelwise, map with V to the internal rank, use U to map back, multiply by original x0, then add residual and bias. Do not replace x0 with xl each layer or broadcast the mask across the batch axis incorrectly. Verify gradient flow instead of using fixed selection.
Illustrative example: Illustration: Rank 32 does not make the output 32-dimensional; it must project back before the elementwise x0 product.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How do rank and channel selection trade expressiveness against DCN-V2's full cross matrix?
Show answer and example
Answer: A full matrix expresses more general transformations. Low rank constrains the subspace while masks select channels adaptively and save capacity budget. More layers need not restore omitted relations. The shared local scorer proxy is not a full parameter-matched DCN-V2 control.
Illustrative example: Illustration: Compressing a thousand dimensions to dozens saves compute but can lose rare geography–category relationships that masks cannot recreate.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why mask the cross input rather than the final score in ML-DCN?
Show answer and example
Answer: Selecting channels before multiplication changes which interactions form. Scaling a final score cannot undo bad crosses. Low rank separately controls internal width. Learned mask values are not causal importance and do not prove low-weight features can be removed everywhere.
Illustrative example: Illustration: Suppressing noisy price before interaction prevents bad combinations; scaling the final prediction suppresses useful signals too.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Core mechanism
Why isn't lower action likelihood after removing an untrusted span a causal safety proof?
CausalArmor: Efficient Indirect Prompt Injection Guardrails via Causal Attribution
Show answer and example
Answer: Leave-one-out ablation shows a change in probability, but removal can change context structure or remove legitimate evidence. Compare user attribution, sanitize only flagged spans, mask contaminated continuation, and repropose. Preserve threshold direction and recomputation; low user attribution alone does not establish an attack.
Illustrative example: If a paragraph contains both a necessary number and a suspicious instruction, its likelihood effect cannot all be attributed to malicious content.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Implementation details
What happens when CausalArmor has no sanitizer or reproposal callback?
CausalArmor: Efficient Indirect Prompt Injection Guardrails via Causal Attribution
Show answer and example
Answer: Withhold the action and mark intervention_required instead of fabricating sanitation or reruns. Mask assistant content after the first hit. Larger τ flags more spans; preserve the sign and require normal authorization for any eventual action.
Illustrative example: Without a real sanitizer, deleting a string is not a successful defense loop.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Method comparison
How does targeted sanitation of flagged tool spans differ from deleting all tool results?
CausalArmor: Efficient Indirect Prompt Injection Guardrails via Causal Attribution
Show answer and example
Answer: Keep unflagged evidence, sanitize flagged spans, and mask potentially contaminated assistant content after the first hit before reproposing. Deleting all tool results also removes legitimate evidence. Reevaluate the new proposal; text removal does not guarantee safety.
Illustrative example: Sanitize an injected instruction and remove affected later drafts while retaining unrelated valid retrieval evidence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Basics
Why does CausalArmor compare user and tool attribution before privileged actions?
CausalArmor: Efficient Indirect Prompt Injection Guardrails via Causal Attribution
Show answer and example
Answer: Untrusted tool content can dominate a proposal. Leave-one-out ablation measures likelihood changes for the same action. If tool attribution exceeds user attribution, selectively sanitize and propose again rather than executing the original action.
Illustrative example: A webpage can suggest deleting files despite no such user request.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
How does two-axis Product-Key Memory in MSN add capacity while reading little knowledge per query?
MSN: A Memory-based Sparse Activation Scaling Framework for Large-scale Industrial Recommendation
Show answer and example
Answer: A slot combines one key from each axis. Summed similarities rank combinations, only a few values are read, and a gate fuses them into the dense backbone. Stored capacity does not all become active compute, and useful routing still matters.
Illustrative example: Illustration: Two sets of 100 keys address 10,000 slots, but one request reads the four most relevant ones.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why preserve unique slot IDs and deduplicate top-k two-axis combinations in MSN?
MSN: A Memory-based Sparse Activation Scaling Framework for Large-scale Industrial Recommendation
Show answer and example
Answer: Both axis addresses identify the slot; the summed similarity is a score, not an ID. Duplicate slots overweight one value. Match value width to the backbone and define short or empty selections. Local genre and transition writes are public proxies.
Illustrative example: Illustration: (2,3) and (3,2) are different slots despite the same index sum, and a duplicate (2,3) is not independent knowledge.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does top-k PKM activation differ from a dense soft mixture over every expert?
MSN: A Memory-based Sparse Activation Scaling Framework for Large-scale Industrial Recommendation
Show answer and example
Answer: A dense mixture usually evaluates all experts before weighting. PKM finds high-scoring key pairs and reads selected values, leaving most slots inactive. Search and memory access still cost time, and hard selection can miss evidence. Total-to-active parameter ratio does not predict measured latency.
Illustrative example: Illustration: Reading four of 10,000 slots differs from running 100 experts, but scattered reads can offset lower FLOPs.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why put capacity in MSN memory slots instead of a fully dense network?
MSN: A Memory-based Sparse Activation Scaling Framework for Large-scale Industrial Recommendation
Show answer and example
Answer: Dense models typically activate all parameters. Memory stores more values but reads few slots, so compute need not grow proportionally. Product keys organize addresses across two axes and a gate fuses values. Sparse reads do not remove maintenance cost or guarantee useful knowledge.
Illustrative example: Illustration: Retrieving four files is cheaper than reading an archive, provided the routing finds the right files.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why tokenize scenario and task in MDL rather than just choosing output heads?
Show answer and example
Answer: Late heads cannot continuously steer feature selection. Tokens let domain queries read relevant fields across layers and task information shape shared representations. Deeper sharing may cause interference; check whether low-traffic settings are overridden by larger ones instead of assuming a unified design helps all tasks.
Illustrative example: Illustration: Home-page clicks and search conversion may need different product evidence. Scenario and task tokens can change early reads instead of splitting one summary at the end.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why label feature, scenario, and task token identities in MDL?
Show answer and example
Answer: Equal final dimensions do not make evidence, conditions, and predictions interchangeable. Keep attention layout and readout positions aligned. Treating scenario count as batch size or reading the wrong task position silently changes the objective. The local 19 scenarios are proxies, not 19 production businesses.
Illustrative example: Illustration: A purchase token cannot supervise viewing just because both are two-dimensional; verify identity mapping as well as shape.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does MDL's deep domain-feature interaction condition more than a shallow gate?
Show answer and example
Answer: A shallow gate adjusts already encoded features. Deep interaction repeatedly conditions representation formation, expanding joint modeling at higher compute and negative-transfer risk. Local accuracy improved with a clear rise in head exposure, not evidence that every group benefited.
Illustrative example: Illustration: The scenario can emphasize price first and combine it with budget next. A shallow gate can only reweight previously formed interactions.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why turn scenario and task into tokens in MDL?
Show answer and example
Answer: They determine which field relations matter, not only which head to use. Domain-feature attention lets scenario queries read features deep in the model, and task tokens and aggregation form specific outputs. Token roles permit interaction but do not ensure intended semantics emerge.
Illustrative example: Illustration: A user's features matter differently for live shopping and short-video viewing. A scenario token can adjust intermediate reads, not just swap the final classifier.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why use headwise SwiGLU after mixing and tokenwise SwiGLU after revert?
TokenMixer-Large: Scaling Up Large Ranking Models in Industrial Recommenders
Show answer and example
Answer: The former learns cross-token combinations from rearranged channels; the latter refines restored per-token states. Residual intervals and intermediate supervision support depth. Rearrangement moves information but does not learn interactions.
Illustrative example: Mix price and preference channels, learn interactions, then restore fields for token-specific processing.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
What can the local RankMixer comparison establish?
TokenMixer-Large: Scaling Up Large Ranking Models in Industrial Recommenders
Show answer and example
Answer: Match width, depth, negatives, and steps, but the result is small single-seed and lacks full attention controls. Bind FFNs to mixed positions and preserve revert order.
Illustrative example: The first mixed head need not be the first original feature slot.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
What does mix–headFFN–revert add over tokenwise FFN?
TokenMixer-Large: Scaling Up Large Ranking Models in Industrial Recommenders
Show answer and example
Answer: Rearrangement exposes slices from different tokens to a nonlinear head transform, then restores layout. It is not dynamic attention, and one small gain cannot replace all attention.
Illustrative example: Each token can receive transformed evidence from other tokens after reverting.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Does parameter-free mixing make the whole TokenMixer-Large parameter-free?
TokenMixer-Large: Scaling Up Large Ranking Models in Industrial Recommenders
Show answer and example
Answer: No. Mix/revert are fixed rearrangements; headwise and tokenwise SwiGLU, deep residuals, and auxiliary losses still learn.
Illustrative example: Channel shuffling is fixed, but the FFNs processing channels train.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Core mechanism
Why not randomly label one click positive when the conversion attribution is only known as a set?
Statistical Learning from Attribution Sets
Show answer and example
Answer: Random pairing invents a precise label, and overlapping sets can affect the same event repeatedly. The method splits binary loss into a label-independent term and a label-dependent moment, estimating risk from set statistics without fabricating individual matches. The printed coefficient for multi-element sets failed a local constant-function check, so the counterexample is retained and the estimator is not fully validated.
Illustrative example: Three impressions may be associated with a purchase and two may also appear in a different attribution set. Choosing one arbitrarily changes the supervision; set statistics also need consistency checks.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Implementation details
Why must an Attribution Sets implementation not silently clip singular denominators or renormalize by the window count?
Statistical Learning from Attribution Sets
Show answer and example
Answer: The binomial tail probability, selection indicator, and fixed denominator together define the paper's estimator. Changing the denominator changes the statistical object. Fail explicitly at singular boundaries and distinguish known population moments from sample plug-in estimates. Keep the multi-element counterexample visible rather than masking it with a numerical fallback.
Illustrative example: Replacing a zero denominator with a tiny positive number returns a finite result but not the paper's estimator. Report that its assumptions do not hold for that input.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Method comparison
How do the assumptions of Attribution Sets risk decomposition differ from maximum-prior pseudo-labeling?
Statistical Learning from Attribution Sets
Show answer and example
Answer: Pseudo-labeling chooses a hidden match before training as if it had full supervision. Risk decomposition keeps the match unidentifiable and uses population moments and a position prior for the label-dependent term. It avoids claiming individual ground truth, but still requires a valid estimator and inputs. The local counterexample is a limited check of the printed coefficient, not a corrected replacement theory.
Illustrative example: A prior that favors the first impression does not make it the true positive. If the set estimator fails the constant-function check, audit it rather than reporting an unbiased result.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Basics
What do Attribution Sets reveal and what do they leave unspecified?
Statistical Learning from Attribution Sets
Show answer and example
Answer: They constrain which clicks may be linked to a conversion, but provide no unique true match; different sets can overlap. Set statistics can support risk estimation, not a definitive label for each click. Keep that distinction separate from true individual attribution.
Illustrative example: Knowing that a purchase could follow any of three impressions does not justify labeling all three as positives or arbitrarily choosing one.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Core mechanism
Why give AIRS-Bench problems, data, and metrics without directly providing baseline solutions?
AIRS-Bench: a Suite of Tasks for Frontier AI Research Science Agents
Show answer and example
Answer: Research involves proposals, implementation, analysis, and revision; complete solutions reduce autonomous design. Distinguish agent, search scaffold, and runtime so outer tree-search gains are not fully attributed to the model. Human SOTA is a comparison, not an answer script.
Illustrative example: Give classification data and scoring rules and let the agent design experiments. Providing the best training code would mostly test script execution.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Implementation details
Does access to an AIRS-Bench scoring script imply access to test labels?
AIRS-Bench: a Suite of Tasks for Frontier AI Research Science Agents
Show answer and example
Answer: No. Submit predictions to isolated test scoring and use only permitted validation feedback for selection. Author results above SOTA on some tasks are not local results; full research training must actually run for a reproduction claim.
Illustrative example: Knowing the metric is accuracy does not reveal any example's correct class.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Method comparison
Why match budgets when comparing sequential MLGym with tree-search AIRA-dojo?
AIRS-Bench: a Suite of Tasks for Frontier AI Research Science Agents
Show answer and example
Answer: Search design affects coverage and cost, and unrestricted branching can consume more evaluations. Fix resources and isolate labels to assess scaffold differences. Metrics from different tasks cannot simply be added.
Illustrative example: A tenfold increase in candidates can yield better models without the gain coming entirely from the search method.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Basics
Is a written method suggestion enough for AIRS-Bench?
AIRS-Bench: a Suite of Tasks for Frontier AI Research Science Agents
Show answer and example
Answer: No. Starting from a problem, dataset, and metric, the agent must propose a method, implement training, analyze experiments, and revise. The agent, search scaffold, and execution harness are distinct. A plausible plan is not a completed experiment.
Illustrative example: Suggesting a different loss is a start; implement and run it, then revise using permitted feedback.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
Why can sequence-level LUSPO updates still need length-bias checks?
Show answer and example
Answer: Aggregation does not remove length coupling in cumulative logp. LUSPO reorganizes scores to avoid systematic length pressure. Short-task normalization checks do not validate long-CoT dynamics.
Illustrative example: Two replies with similar per-token changes can have different cumulative ratios solely because of length.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
Why don't local short arithmetic tasks establish long-CoT length stability?
Show answer and example
Answer: Generation, sequence ratios and normalization run, but short replies do not cover long reasoning dynamics. Use effective response length without padding/prompt.
Illustrative example: A dozen-word answer doesn't test drift across thousands of tokens.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
How does the LUSPO sequence ratio differ from tokenwise clipping?
Show answer and example
Answer: Exponentiate length-normalized sequence log-ratio and share the signal over the response. Tokenwise clips act locally, with different dynamics. Sequence aggregation alone doesn't guarantee long-length stability.
Illustrative example: An increased token ratio and a decreased one can cancel in a sequence but clip differently.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
Why examine response-length trajectories in LUSPO?
Show answer and example
Answer: RLVR objectives implicitly weight length differently, potentially causing collapse or drift. Normalize sequence logp as prescribed to separate reward learning from length scale.
Illustrative example: Stable accuracy with shrinking replies can still lose long-reasoning coverage.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why does DOS orthogonally rotate content vectors before residual quantization?
DOS: Dual-Flow Orthogonal Semantic IDs for Recommendation in Meituan
Show answer and example
Answer: Original directions can concentrate information in a few coordinates. Rotation redistributes directions while preserving geometry; residual quantization allocates coarse and fine information, with both behavior and content calibrating the codebook. Orthogonality does not make semantics independent or prove recommendation utility.
Illustrative example: Rotation can spread concentrated coordinate variance, but only behavioral targets show whether it separates attributes users care about.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
Why check both orthogonality and reconstruction errors in DOS?
DOS: Dual-Flow Orthogonal Semantic IDs for Recommendation in Meituan
Show answer and example
Answer: `WᵀW` near identity verifies rotation only. Reconstruction checks residual-code approximation; good values on both still do not prove retrieval quality. Use the same rotation version for training and evaluation and document mapping changes before reusing codes.
Illustrative example: A perfectly orthogonal W may align points poorly with the codebook, so near-zero orthogonality error cannot replace reconstruction or ranking scores.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
Why constrain DOS projection to be orthogonal rather than freely learned?
DOS: Dual-Flow Orthogonal Semantic IDs for Recommendation in Meituan
Show answer and example
Answer: Orthogonal transforms preserve distances and norms while reorienting the space for quantization. General projections can distort scales. Constraints reduce freedom but do not minimize quantization error or ensure independent semantic factors. The local Procrustes path is a specific controlled version.
Illustrative example: Rotating axes preserves pairwise distance, unlike stretching an axis, but a better grid orientation does not causally decompose preferences.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why do DOS behavior and semantic streams shape one codebook together?
DOS: Dual-Flow Orthogonal Semantic IDs for Recommendation in Meituan
Show answer and example
Answer: Reconstruction alone can miss collaborative distinctions. Behavior provides consumption relations and content preserves semantic structure. Balance both to avoid popularity overwhelming details; co-occurrence is not semantic identity.
Illustrative example: Similar-topic movies with different audiences can separate, while different genres sharing travel interests may gain useful links.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why are controller memory selection and designer skill compilation different jobs?
Show answer and example
Answer: Selection decides which experiences the current task reads. Compilation turns recurring successes into reusable procedures on a longer timescale. Feedback also drives versions. One retrieved success does not establish that every detail should be consolidated; distinguish reusable steps from incidental conditions.
Illustrative example: Example: Retrieve a prior fix for today's error and compile a skill only after confirming a repeated pattern. Do not hard-code a temporary account from one log.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
What is the evidence boundary between local skill-version updates and full designer learning?
Show answer and example
Answer: The local run verifies selection, compilation-style transitions, versions, and budgets without the paper's LLM, RL, or real environment. Version increments show the path ran, not reliable skill extraction from natural-language logs. Perfect controlled scores are not paper cross-task ability.
Illustrative example: Example: Rules can convert recurring steps to a template for a pipeline test. Without a learned designer, do not claim open-task procedure discovery.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
Why is repeated success stronger evidence for consolidation than a single successful trace?
Show answer and example
Answer: One success may depend on chance or hidden help. Repetition supports a pattern across instances and makes it a better skill candidate. It does not prove causal necessity; inspect prerequisites, failures, and version changes. Frequency does not replace validation.
Illustrative example: Example: Three successes clear cache first because those tasks may all need it. Record the condition rather than mandate clearing cache everywhere.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Why turn one-off experiences into callable operational knowledge in MemSkill?
Show answer and example
Answer: Episodic memory records what happened, while later tasks may need repeatable steps, preconditions, and recovery. MemSkill extracts skills and lets execution select relevant memories. Over-abstraction can lose conditions, and repeated success is not universal applicability. Check transfer through actual execution.
Illustrative example: Example: A spreadsheet repair includes long dialogue. Future work needs check column names before date conversion and the conditions where that rule applies.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Core mechanism
Why should research-search node value include both performance and runtime?
MARS: Modular Agent with Reflective Search for Automated AI Research
Show answer and example
Answer: Under a fixed training budget, an equally good but slower option crowds out later exploration. MARS normalizes execution metrics, adds a time penalty, and bounds improvement or debugging. Reflection uses code diffs and real results, not the model's predicted performance in place of node measurements.
Illustrative example: If two options reach similar validation scores but one takes ten times longer, prioritize the faster one to test more structural hypotheses.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Implementation details
What boundaries apply to identical node metrics, debugging, and experience capacity?
MARS: Modular Agent with Reflective Search for Automated AI Research
Show answer and example
Answer: If the metric range is zero, normalize to 0.5 rather than divide by zero. The paper caps search at ten debugging attempts, two improvement children, and thirty recent experiences. There is no local implementation or training. Author code and released traces do not establish a fully reproducible controller, and copying a reward formula is not research capability.
Illustrative example: If all AUC values tie, do not produce NaN or assign an arbitrary candidate a very high score. The common midpoint leaves factors such as time active.
Sections in the Chinese source: 核心公式, 本地复现与成本判断. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Method comparison
What do the new, improve, and debug search actions change?
MARS: Modular Agent with Reflective Search for Automated AI Research
Show answer and example
Answer: New explores a different research proposal; improve builds on a valid node; debug repairs a candidate that cannot execute validly. They produce different evidence: fewer errors are not higher metrics, and executability does not justify more search. MARS selects by execution status and budget and caps debugging and descendants. Estimated scores for failures are not measurements.
Illustrative example: Debug an import error before measuring performance. Improve a valid but promising score; create a new branch to try a wholly different approach.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Basics
Why does Design–Split–Implement first produce a modular repository?
MARS: Modular Agent with Reflective Search for Automated AI Research
Show answer and example
Answer: Research changes touch several components. Defining design and module boundaries before implementation lets branches improve locally and reuse debugging experience instead of rewriting everything. Interfaces can still be incomplete; actual training establishes behavior, not tidy structure or small diffs.
Illustrative example: Separate loading, model, and evaluation so a branch can replace only the model under an unchanged evaluation entry point, without confounding data changes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why retrieve a searchable catalog before multi-objective preference alignment in RAG-QAC?
Show answer and example
Answer: Free generation can create plausible suggestions with no results. Catalog retrieval grounds suggestions in traceable candidates, then SFT/DPO balance relevance, safety, engagement, context, and diversity. Alignment cannot create missing inventory, and engagement should not trump factuality.
Illustrative example: Illustration: Retrieve real tents and cookware for 'light camping' before suggesting a nonexistent model.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Why keep RAG-QAC retrieval evidence separate from gold completion labels?
Show answer and example
Answer: Evidence comes from the prefix and catalog, while gold is training supervision. Retrieving from the correct held-out completion leaks it. Verify catalog and context independently rather than treating membership as all quality checks.
Illustrative example: Illustration: Inserting the held-out tent model before generating lets the model repeat the answer rather than retrieve from the prefix.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
What do SFT and DPO add beyond a retrieval-template system for RAG-QAC?
Show answer and example
Answer: Templates ground some terms but fix expression and priorities. SFT learns contextual completions and DPO compares relevance, safety, engagement, and diversity. Goals can conflict; avoid losing truth for clicks. A fixed local scorer does not prove language post-training.
Illustrative example: Illustration: Learn a trade-off between two searchable suggestions, one closer to the user and one covering a new direction.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Why assess catalog groundedness separately from contextual groundedness in RAG-QAC?
Show answer and example
Answer: Catalog checks whether a suggestion maps to real searchable content; context checks fit with prefix and known needs. Each can pass alone and neither guarantees safety or engagement.
Illustrative example: Illustration: Real adult hiking boots fail context for children's rain boots, while an invented perfect child model fails the catalog.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why compress heterogeneous features into wide Prime Tokens and then enhance each separately?
Zenith: Scaling up Ranking Models for Billion-scale Livestreaming Recommendation
Show answer and example
Answer: Token Fusion trades fewer positions for wider channels; RSA/TMHSA maintain differentiation and SwiGLU or MoE adds role-specific transformations. Check information loss and matched compute.
Illustrative example: Reduce pairwise interactions over 100 fields while preserving meaningful user/item roles.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Does local Zenith establish a scaling law?
Zenith: Scaling up Ranking Models for Billion-scale Livestreaming Recommendation
Show answer and example
Answer: No. Small fixed data exercises fusion, differentiated interaction, and enhancement without production hardware or size. A scaling slope needs matched multi-size curves.
Illustrative example: A win at one scale does not imply the gap grows with size.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does token-specific TSwiGLU differ from shared FFNs?
Zenith: Scaling up Ranking Models for Billion-scale Livestreaming Recommendation
Show answer and example
Answer: Independent transformations let roles specialize, but add parameters. Match budgets before attributing gains to architecture.
Illustrative example: User and item slots may benefit from different nonlinearities.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why fuse many features into a few Prime Tokens?
Zenith: Scaling up Ranking Models for Billion-scale Livestreaming Recommendation
Show answer and example
Answer: Fewer wider tokens reduce interaction count and retain capacity; differentiated attention and independent enhancement counter homogenization. Compression can still lose detail.
Illustrative example: Put sparse fields in several wide slots without making every slot identical.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why does OneMall need scenario conditioning and multi-behavior cross-attention after unifying Semantic IDs?
Show answer and example
Answer: One output vocabulary gives each item one identity across scenes, not one behavior meaning. Scenario conditions specify the task and cross-attention selects pertinent history. Query-Former compression and sparse experts manage context and capacity. A unified model family still needs scene-specific behavior.
Illustrative example: Illustration: The same jacket can be clicked in a card or ordered after a livestream. Its ID is shared, but live recommendation needs different behavior evidence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Why don't local three-level SIDs and a scene prompt prove OneMall Query-Former, MoE, and RL were reproduced?
Show answer and example
Answer: The local path adds scene conditions, residual codes, and cross-behavior fusion. Query compression, actual routing, and policy updates need separate evidence. Report the small NDCG increase and Hit decrease within that scope rather than invoking the full paper family.
Illustrative example: Illustration: Three emitted item codes do not imply experts. Naming a loss 'reward' does not establish RL without rollout and updates.
Sections in the Chinese source: 本地复现. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
How does OneMall's sparse MoE change capacity and specialization relative to a dense generator?
Show answer and example
Answer: Experts add conditional capacity while sparse activation limits per-request compute. Dense sharing is simpler but can favor frequent scenes. Experts can starve or duplicate, so total parameters are not effective capacity. Local SID and scene fusion do not reproduce full MoE/RL.
Illustrative example: Illustration: Live shopping and video can use different expert mixtures. If the gate always uses one expert, unused experts add no specialization.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Why are shared item codes useful across OneMall cards, short videos, and livestreams?
Show answer and example
Answer: Shared identity avoids fragmented mappings and separate modeling. Scene conditions and multi-behavior evidence distinguish consumption contexts. The code does not equalize exposure goals, video completion, commerce, or watch time, and transfer can conflict.
Illustrative example: Illustration: Jacket videos and cards share a code, but video completion and orders should not become equally strong positives.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Core mechanism
How does ME-POIs transfer time information to sparse places?
Mobility-Embedded POIs: Learning What A Place Is and How It Is Used from Human Movement
Show answer and example
Answer: High-frequency anchors contribute within each spatial scale using distance normalization; then average scales so more anchors do not dominate. Trajectories connect recent visits and long-term prototypes; text supplies distinct semantic supervision.
Illustrative example: Normalize two local anchors and 20 broader anchors separately before averaging their predictions.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Implementation details
Should repeated visits duplicate a place prototype as a negative?
Mobility-Embedded POIs: Learning What A Place Is and How It Is Used from Human Movement
Show answer and example
Answer: No. Eq. 9 uses distinct minibatch place prototypes. Keep temporal KL in direction r||q and sum across places as specified.
Illustrative example: Three visits to a POI still supply one prototype.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Method comparison
How does per-scale normalization differ from one softmax over anchors?
Mobility-Embedded POIs: Learning What A Place Is and How It Is Used from Human Movement
Show answer and example
Answer: Normalize each scale independently before averaging. A pooled denominator changes relative scale weights based on candidate mass or count.
Illustrative example: Near and far scales each contribute one prediction regardless of candidate count.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Basics
Why combine trajectories, prototypes, and text?
Mobility-Embedded POIs: Learning What A Place Is and How It Is Used from Human Movement
Show answer and example
Answer: Sparse locations lack visits. Short trajectories connect to long-term prototypes, high-frequency anchors transfer temporal patterns, and text supplies semantic alignment.
Illustrative example: A rarely visited place can borrow time-of-day patterns from nearby busy places.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Core mechanism
How does ECO preserve small updates without high-precision masters?
ECO: Quantized Training without Full-Precision Master Weights
Show answer and example
Answer: Update quantized weights and feed requantization error into existing momentum without a separate persistent buffer. Compensation depends on step size, elementwise for Adam. The buffer-free derivation is approximate, not exact high-precision equivalence.
Illustrative example: A sub-grid update can leave weights unchanged but influence momentum and later steps.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Implementation details
Why can't ECO use the same scalar coefficient when moving from SGDM to Adam?
ECO: Quantized Training without Full-Precision Master Weights
Show answer and example
Answer: Bias correction and second moments create elementwise effective η_t in Adam; use that for error injection. Record quantization format and scale granularity.
Illustrative example: Two coordinates with different preconditioning cannot share a scalar compensation denominator.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Method comparison
Is buffer-free compensation exactly equivalent to storing full adjacent errors?
ECO: Quantized Training without Full-Precision Master Weights
Show answer and example
Answer: No. It uses an approximation that neighboring errors are close, with separate convergence analysis. It is not exact under arbitrary quantization.
Illustrative example: Large differences between adjacent errors can invalidate the approximation.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Basics
Why remove both master weights and a dedicated error buffer?
ECO: Quantized Training without Full-Precision Master Weights
Show answer and example
Answer: Both consume persistent memory. ECO injects error into existing momentum to accumulate small updates without storing them separately.
Illustrative example: An update too small for one quantization step can still affect later optimization.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Core mechanism
Why give sparse EASQ survey satisfaction a separate LoRA path and dynamic reference?
Towards End-to-End Alignment of User Satisfaction via Questionnaire in Video Recommendation
Show answer and example
Answer: Abundant behavior labels can swamp direct but sparse satisfaction signals. A separate low-rank path preserves survey expertise; online preference optimization uses it as a changing reference instead of chasing clicks alone. This reduces interference, not survey selection bias.
Illustrative example: Curiosity clicks can coexist with dissatisfaction in a few surveys, which remain influential through a dedicated expert.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Implementation details
Why verify actual trainable parameters for EASQ rather than checking only a LoRA configuration?
Towards End-to-End Alignment of User Satisfaction via Questionnaire in Video Recommendation
Show answer and example
Answer: If the optimizer updates the backbone, the run is not an independent low-rank path for sparse satisfaction. Reference probabilities must match recorded versions, and serving dependencies must agree with training. A public proxy without surveys is not satisfaction reproduction.
Illustrative example: Full backbone updates can lower loss without proving LoRA protected ranking; inspect gradients and optimizer groups.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Method comparison
How does a dynamic survey reference differ from a fixed SFT reference?
Towards End-to-End Alignment of User Satisfaction via Questionnaire in Video Recommendation
Show answer and example
Answer: The fixed reference anchors to the initial distribution, while the survey expert adapts to newer satisfaction evidence. A moving reference is nonstationary and needs versions and limits. Updates need not always improve it, and LoRA alone does not prove backbone retention.
Illustrative example: A newer expert can reflect growing dislike of repetitive content, while a fixed reference may keep rewarding stale behavior.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Basics
Why separate survey satisfaction and behavior feedback parameter paths in EASQ?
Towards End-to-End Alignment of User Satisfaction via Questionnaire in Video Recommendation
Show answer and example
Answer: Behavior is plentiful but can reflect short-lived clicks; surveys are direct but sparse. Separate LoRA protects satisfaction learning and supplies a preference reference. Survey response bias remains, so these are not unbiased population labels.
Illustrative example: Clicks followed by reported regret can be drowned out in pooled updates but remain influential through an independent expert.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
Why jointly reward tool chains and outcomes even with separate Planner and Executor roles?
Show answer and example
Answer: A well-formed plan does not ensure tools exist, order is executable, or the answer is correct. An executor can supply bad arguments despite a sound plan. Offline trial and error records usage and failures; dense online signals separately constrain tool choice, order, and result. A high process component is not actual success.
Illustrative example: Looking up an order before refunding is a correct sequence, but a wrong order ID still fails. Reward both chain quality and final outcome.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
Why retain PEARL tool/order/success components rather than only total reward?
Show answer and example
Answer: Each component measures tool identity, ordering, or completion, helping localize planning signals and check weight directions. A total hides defective components. Local exploration and reuse are not live ToolHop API calls or complete GRPO.
Illustrative example: If tools are correct but ordered incorrectly, the order term should explain the lower reward.
Sections in the Chinese source: 核心公式, 复现边界. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
How does training feedback differ between decoupled Planner/Executor and one model thinking and calling tools as it goes?
Show answer and example
Answer: Decoupling can use tool choice, order, and final success as planning rewards while the Executor handles execution feedback. A good plan does not guarantee success: verify arguments and runtime effects.
Illustrative example: A plan can pick all three tools correctly yet fail because the Executor supplies wrong arguments.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why does PEARL run offline tool trial and error before online planning?
Show answer and example
Answer: Hallucinated tools and wrong arguments break multi-hop plans. Offline exploration collects tool usage and failure conditions, then online separates Planner and Executor so planning can use more reliable experience.
Illustrative example: Learn required parameters and failure conditions before the Planner decides a call sequence.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why does LLaTTE still read recent history with candidate-aware attention after caching LLM user representations?
LLaTTE: Scaling Laws for Multi-Stage Sequence Modeling in Large-Scale Ads Recommendation
Show answer and example
Answer: Async long-history encoding supports more compute but refreshes periodically and compresses details. The online short sequence adds recent behavior, and the candidate query picks relevant evidence. Stable knowledge and real-time relevance are distinct responsibilities. Measure transfer of upstream gains downstream instead of assuming all of them survive.
Illustrative example: Illustration: An overnight cache remembers photography, but a new backpack search signals travel. Online history adds the new intent and lets the current candidate read relevant knowledge.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why can't LLaTTE's local four latent queries and recent-12 path establish the value of a 2,000-dimensional production cache?
LLaTTE: Scaling Laws for Multi-Stage Sequence Modeling in Large-Scale Ads Recommendation
Show answer and example
Answer: The small setup trains long summaries, short history, and gating but does not fully measure production refreshes, roughly 400 online events, or multisource features. Verify gradient flow and candidate dependence, then report the matched short baseline and head exposure. Size differences cannot become measured QPS or conversion gains.
Illustrative example: Illustration: Four summary vectors reading 24 movie events test information flow, not the latency of a production cache over tens of thousands of multisource events.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
Why isn't LLaTTE's transfer efficiency τ just the upstream NE improvement?
LLaTTE: Scaling Laws for Multi-Stage Sequence Modeling in Large-Scale Ads Recommendation
Show answer and example
Answer: A better upstream model need not yield proportional downstream gains: cache size, fusion capacity, and task objectives can discard information. τ compares the two improvements to help decide where to spend compute. A tiny denominator or mismatched protocols makes the ratio unstable; an empirical trend is not a theorem.
Illustrative example: Illustration: If upstream NE improves and downstream NE barely changes, candidate-aware fusion may be more useful than adding upstream parameters.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why scale LLaTTE's async long history separately from the online short sequence?
LLaTTE: Scaling Laws for Multi-Stage Sequence Modeling in Large-Scale Ads Recommendation
Show answer and example
Answer: The upstream model processes more history and caches an overview. The downstream model uses fresh events and candidate-aware interactions without rerunning the big model on each request. Each stage has a bottleneck, so measure compression and transfer efficiency rather than assuming all upstream gains reach serving.
Illustrative example: Illustration: A cached profile retains years of interests, while recent baby-product browsing can change ranking immediately instead of waiting for the next refresh.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Core mechanism
Why doesn't failing on the original bug and passing on a generated patch prove the patch correct?
Dynamic Cogeneration of Bug Reproduction Test in Agentic Program Repair
Show answer and example
Answer: A self-generated test can cover only a local behavior the patch happens to satisfy. Agreement does not establish full semantics. This defines a candidate BRT; stronger evidence needs isolated reference repairs and other verification. Grouping votes by non-test patch keeps different tests from splitting the same fix.
Illustrative example: Example: A patch returns a constant and its test checks only one matching input. Both conditions pass, but other inputs remain wrong.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Implementation details
Can any execution exception count as a successful bug reproduction?
Dynamic Cogeneration of Bug Reproduction Test in Agentic Program Repair
Show answer and example
Answer: No. A nonempty assertion must raise assertion_failed on the original, then pass on the fix with acceptable regression results. Syntax errors and unrelated exceptions do not qualify. Execution uses a bounded AST allowlist and timed isolated process, not an arbitrary hostile-code sandbox. Real LLM reviewers and private Google bugs are not run.
Illustrative example: Example: An import error does not reproduce a business-logic bug just because the patched run imports normally.
Sections in the Chinese source: 核心公式, 本地复现, 代码映射与边界. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Method comparison
Why group candidates by non-test patch instead of counting each generated test separately?
Dynamic Cogeneration of Bug Reproduction Test in Agentic Program Repair
Show answer and example
Answer: One fix can pair with several tests. Pairwise votes can split or inflate support for the same code. Group by non-test changes, then choose a test within the majority fix group. Identifier normalization is a heuristic, not a proof of semantic equivalence.
Illustrative example: Example: Ten traces with the same code and different assertions are one repair group, not ten independent fixes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Basics
Why is an executable bug-reproduction test valuable before automated repair?
Dynamic Cogeneration of Bug Reproduction Test in Agentic Program Repair
Show answer and example
Answer: Text issues can be ambiguous. A test makes failure observable and checks whether a patch changes the intended behavior. Dynamic BRT generates and selects tests during repair search. They are local evidence and may lack coverage or overfit a bad patch; they cannot replace full correctness checks.
Illustrative example: Example: Reproduce an empty-list crash with a fixed input. After that test passes, independently check other inputs.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why insert a thinking token before each SID stage and align it to codebooks?
S2GR: Stepwise Semantic-Guided Reasoning in Latent Space for Generative Recommendation
Show answer and example
Answer: Early coarse codes constrain later paths. Intermediate states receive contrastive codebook supervision; collaborative graphs and balancing add behavior and usage signals. This need not be faithful natural-language reasoning.
Illustrative example: Align the intermediate outdoor-category state before selecting finer camping gear codes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Are the local S2GR codebook and thinking states the production model?
S2GR: Stepwise Semantic-Guided Reasoning in Latent Space for Generative Recommendation
Show answer and example
Answer: No. Fixed MovieLens data tests core mechanisms. Align current code, earlier codes, and thinking position; never tune balance using test catalog labels. Production gains are author-reported.
Illustrative example: Uniform use in a small catalog does not validate a production hierarchy.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
What do thinking alignment and codebook load balancing constrain?
S2GR: Stepwise Semantic-Guided Reasoning in Latent Space for Generative Recommendation
Show answer and example
Answer: Alignment links intermediate states to code semantics. Load balancing prevents concentrated code use. Uniform use does not establish semantics, and semantic alignment does not ensure balance.
Illustrative example: Randomly spreading examples can fix utilization while losing meaning.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Is S2GR's thinking token a long free-form natural-language chain?
S2GR: Stepwise Semantic-Guided Reasoning in Latent Space for Generative Recommendation
Show answer and example
Answer: It is an intermediate state before each SID level, aligned to that level's codebook. SIDs remain a discrete hierarchical path; the name does not prove natural-language reasoning.
Illustrative example: Insert a state, choose a level code, and move on without emitting a prose essay.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Core mechanism
Why must OPSD student rollouts not see the verified solution?
Show answer and example
Answer: The teacher uses it to score actual student prefixes; student stays in deployment conditions. Pointwise clipping bounds stylistic outliers. Giving student the answer bypasses learning. Local checks cover mechanism only.
Illustrative example: Teacher can evaluate a half-completed derivation knowing the answer; student cannot use that answer during independent generation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Implementation details
Why clip OPSD pointwise rather than one response coefficient?
Show answer and example
Answer: Limit per-token divergence so unusual stylistic positions cannot dominate. Freeze teacher target and align masks. Local views aren't Qwen competition-math training.
Illustrative example: One anomalous format token needn't amplify the entire sequence's guidance.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Method comparison
How does OPSD isolation differ from answer-conditioned SFT?
Show answer and example
Answer: Hide solution during student sampling and score its states with privileged teacher. Adding solution to student would teach unavailable deployment dependence. Full distribution distillation isn't imitation of one answer.
Illustrative example: A useful intermediate expression can appear for teacher but not as a shortcut in student prompt.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Teachers and correction: when extra context can help
Basics
Why can the same model be a more informed OPSD teacher?
Show answer and example
Answer: Teacher additionally sees a verified solution, while student gets problem only. The difference is information, not model size. It depends on reliable solutions.
Illustrative example: Judging with the conclusion known differs from solving it independently.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why combine time-adaptive chunks with global, transition, and local attention?
Unleashing the Potential of Sparse Attention on Long-term Behaviors for CTR Prediction
Show answer and example
Answer: Equal event counts may span minutes or months. Time grouping respects personal rhythm; three routes capture overview, shifts, and fine details. Relative time encodes intervals/cycles but can miss sudden or sparse interests.
Illustrative example: Separate a furniture browsing burst from later supply shopping while modeling the transition.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Do local time features replace production chronology?
Unleashing the Potential of Sparse Attention on Long-term Behaviors for CTR Prediction
Show answer and example
Answer: No. Fixed MovieLens tests chunks and biases only. Sort pre-request events, align biases with valid masks, and exclude future labels.
Illustrative example: A shared wrong threshold can impose identical chunks on users with different rhythms.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does three-path EvoAttention differ from a sparse local window?
Unleashing the Potential of Sparse Attention on Long-term Behaviors for CTR Prediction
Show answer and example
Answer: Local reads details, global chunks represent cross-period interests, and transition paths model changes. Time biases capture intervals and calendar cues, not causal guarantees.
Illustrative example: Weekend patterns can correlate with interest without causing clicks.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why chunk histories using personal time gaps?
Unleashing the Potential of Sparse Attention on Long-term Behaviors for CTR Prediction
Show answer and example
Answer: Event counts do not separate dense sessions from long breaks. Adaptive chunks follow individual pace, with global, transition, and local attention handling different scales.
Illustrative example: A burst today and a revisit next month should not be grouped solely because events are adjacent.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why does HCUB inherit parent rewards in sparse contexts while preserving exploration?
Hierarchical Contextual Uplift Bandits for Catalog Personalization
Show answer and example
Answer: Fine-grained estimates have high variance. Parent rewards give a stable starting point, and local data gradually gain weight. Choose by incremental value plus uncertainty to avoid early lock-in. Hierarchical sharing can mask minorities, so do not replace all local evidence with parent averages.
Illustrative example: A sparse device group borrows regional experience until feedback reveals its own preferences, with exploration helping discover that difference.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
How should HCUB avoid propagating one sparse child's feedback to every node?
Hierarchical Contextual Uplift Bandits for Catalog Personalization
Show answer and example
Answer: Define local statistics and parent prior update scopes. Reading a mixture does not copy one outcome across the tree. Use defined η and uncertainty, retain action identities, and do not invent failures for unobserved arms.
Illustrative example: Update a child's observed success and permitted ancestor aggregates without marking every sibling as a success.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
How does HCUB incremental reward plus UCB differ from greedy absolute CTR?
Hierarchical Contextual Uplift Bandits for Catalog Personalization
Show answer and example
Answer: Increment measures change relative to a reference, and the exploration bonus addresses uncertainty. CTR greedy favors already responsive groups. Increment needs suitable causal data; confidence bonuses are not identification, and movie proxies do not establish interventions.
Illustrative example: An active group might click anyway, while a less active group may respond more to the action.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why choose context-tree granularity dynamically in HCUB?
Hierarchical Contextual Uplift Bandits for Catalog Personalization
Show answer and example
Answer: Fine nodes are relevant but sparse; parents are better sampled but coarse. Reward inheritance handles cold starts and yields to local data. Tree design and mixing affect bias, so more partitions do not ensure accuracy.
Illustrative example: Borrow a parent estimate after two events, then rely more on the child after 100 observations.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Core mechanism
Why doesn't an SQL scorer computing definitive answers implement LifeAgent tool reasoning?
LifeAgentBench: Benchmarking LLMs for Long-Horizon, Cross-Dimensional Lifestyle Health Reasoning
Show answer and example
Answer: The scorer uses rules and records to verify final answers; it does not decompose tasks, retrieve evidence, maintain statistics, or revise plans. Keep scorer and tool baseline distinct and do not let actors read reference calculations. Field checks are not long-term health-record reasoning; this describes evaluation, not health advice.
Illustrative example: A verifier can compute mean weekly sleep, but the agent must choose the window, query records, and check missing values without calling the reference-answer function.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Implementation details
How do read-only SQL and Context Prompting isolate gold answers?
LifeAgentBench: Benchmarking LLMs for Long-Horizon, Cross-Dimensional Lifestyle Health Reasoning
Show answer and example
Answer: For SQL, the model generates an allowed query before receiving results. Context Prompting selects public evidence by user first. Gold stays with the scorer. Numeric grading follows piecewise tolerances, and multi-item answers must match item counts, not merely substrings.
Illustrative example: A correct term accompanied by extra incorrect items is not a fully correct answer.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Method comparison
How do executable SQL, complete evidence, and final correctness differ?
LifeAgentBench: Benchmarking LLMs for Long-Horizon, Cross-Dimensional Lifestyle Health Reasoning
Show answer and example
Answer: Executability concerns syntax and runtime. EX measures evidence completeness on its defined denominator, and Acc|EX checks answers given complete evidence. Mixing denominators conceals retrieval or reasoning problems.
Illustrative example: A query can execute but omit a required user and fail to collect all evidence.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Basics
Why do LifeAgentBench's long-term life records require more than classifying one event?
LifeAgentBench: Benchmarking LLMs for Long-Horizon, Cross-Dimensional Lifestyle Health Reasoning
Show answer and example
Answer: Questions span times, dimensions, or users across diet, sleep, activity, and mood records. Gather relevant evidence before calculating or comparing. Filtering and aggregation scope are part of the answer; a single record rarely suffices.
Illustrative example: To compare activity and sleep for one user over a week, use both kinds of records for the same dates. One day's step count is incomplete.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why does HyFormer apply Query Boosting after every history read and pass the queries back to the next layer?
HyFormer: Revisiting the Roles of Sequence Modeling and Feature Interaction in CTR Prediction
Show answer and example
Answer: If history is compressed once before field interactions, those interactions cannot change what evidence is read next. HyFormer alternates sequence reads and non-sequence token mixing, letting new evidence enter field interactions and boosted queries return to history. The compact queries and field representations co-evolve; this does not require rewriting the raw history for every candidate.
Illustrative example: Illustration: The first layer finds an outdoor interest. After mixing in rainy weather and item material, the next query can focus on waterproof gear. A one-shot summary followed by an MLP cannot easily ask that new question of the history.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why must HyFormer feed the boosted query back into the next cross-attention layer?
HyFormer: Revisiting the Roles of Sequence Modeling and Feature Interaction in CTR Prediction
Show answer and example
Answer: Reusing the original Q would make each layer repeat the same question and eliminate co-evolution. Sequence K/V, decoded queries, and non-sequence tokens also need aligned layouts; per-token transforms operate on the mixed layout. The local two-query reduction is a specific implementation, not an exact match for production parameters and cache paths.
Illustrative example: Illustration: The second query should carry the first layer's budget and use-case information. If code accidentally reuses Q0, a deeper model can still reduce loss without implementing the paper's iterative information loop.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
When do interactions occur in HyFormer's query co-evolution versus LONGER followed by RankMixer?
HyFormer: Revisiting the Roles of Sequence Modeling and Feature Interaction in CTR Prediction
Show answer and example
Answer: The serial path summarizes the sequence before field mixing, so the initial history read cannot react to later field results. HyFormer sends boosted queries back for another read, conditioning it over several layers. Earlier interaction adds expressiveness but may also strengthen popularity bias: the source reports a large local increase in head share, not unequivocally better personalization.
Illustrative example: Illustration: One round finds camping signals, field mixing adds a budget constraint, and the next round seeks lightweight gear. A completed one-shot summary generally cannot reread raw history under the new constraint.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why initialize HyFormer's Global Queries from non-sequence fields and the sequence mean?
HyFormer: Revisiting the Roles of Sequence Modeling and Feature Interaction in CTR Prediction
Show answer and example
Answer: The query needs the current global context and a history overview so it can read the long sequence purposefully instead of starting from an unconditional slot. Each decoding step is followed by boosting to combine fields with history. The initial summary can lose details; later reads can help but cannot guarantee complete recovery.
Illustrative example: Illustration: A travel setting and a photography-heavy history can steer the initial query toward lightweight camera needs. A constant query sees the current setting only in a later layer.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
Why constrain decision probability if accuracy improves while ECE worsens?
Show answer and example
Answer: Correctness only needs the top class right; calibration requires confidence to match actual hit frequency. RLVR can make correct choices overconfident. Measure separately, choose T on validation and reserve test. Post-hoc calibration is not full RL reproduction.
Illustrative example: Two models with identical labels can assign near-certainty or moderate confidence and differ in probability quality.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
May calibration T be selected on test NLL?
Show answer and example
Answer: No. Fit validation NLL, freeze, then report test accuracy and calibration. Choosing on test biases final evaluation. Local coverage is calibration and split isolation.
Illustrative example: Trying all temperatures on test and reporting the best overstates generalization.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
How does local temperature calibration differ from full RLVR training?
Show answer and example
Answer: Temperature rescales existing logits; full training changes the decision model. Shared calibration goals do not make postprocessing a full reproduction.
Illustrative example: A frozen model with tuned T uses no resampled trajectories or policy gradient.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
Does calibration-aware RL score the entire explanation through its decision token?
Show answer and example
Answer: No. It studies final-decision probability and calibration, not factual fidelity of all reasoning. Correct choices and credible probabilities require separate measurement.
Illustrative example: High probability for A doesn't make every preceding explanation step correct.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Core mechanism
Why can't editing rubric strings or averaging supplied scores reproduce CoReflect's reflective co-evolution?
CoReflect: A Reflective Co-Evolution Framework for Improving Conversational Evaluation
Show answer and example
Answer: The core needs semantic planning of scenarios, dialogue simulation, evaluation, and an analyzer that uses reasons to identify deficits and update both tasks and criteria. Mechanical state transitions are scaffolding without semantic judgment or real interaction. Source formulas explain the loop, not local execution.
Illustrative example: An analyzer should identify missed empathy criteria and create relevant scenarios. Manually changing a weight from one to two does not provide that reflection.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Implementation details
Did the local project implement CoReflect's 953 scenarios and full semantic modules?
CoReflect: A Reflective Co-Evolution Framework for Improving Conversational Evaluation
Show answer and example
Answer: No. This is literature coverage without a heuristic adapter. JSON state transitions or averaging supplied values do not replace LLM checks and dialogue generation; formulas explain the source method.
Illustrative example: More loop iterations do not show the evaluator learned a new judgment dimension.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Method comparison
How does CoReflect persona–scenario consistency checking differ from valid JSON?
CoReflect: A Reflective Co-Evolution Framework for Improving Conversational Evaluation
Show answer and example
Answer: Syntax only makes fields readable, not semantically compatible. The paper retains pairs passing actual LLM consistency checks. This filter is part of task quality; a state machine or manual score does not replace semantic checking and reflection.
Illustrative example: A persona unable to use the internet paired with a completed online booking can be valid JSON and still require semantic review.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Basics
What do CoReflect's planner, simulator, evaluator, and analyzer produce?
CoReflect: A Reflective Co-Evolution Framework for Improving Conversational Evaluation
Show answer and example
Answer: The planner organizes tasks, the simulator generates interactions, the evaluator applies current criteria, and the analyzer distills reflection that changes later tasks and criteria. These are semantic model roles, not four JSON fields. Local work did not substitute handwritten rules for them.
Illustrative example: An ambiguous criterion appears during dialogue. The analyzer identifies it, then later rounds revise the rubric and plan scenarios that test the issue.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Core mechanism
Why is Engram conditional memory rather than another set of MoE experts?
Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models
Show answer and example
Answer: MoE conditionally executes networks; Engram hashes normalized n-grams to deterministic table lookups and gates them into state. Stored patterns complement compositional compute, but memory size, collisions, bandwidth, and fusion cost remain.
Illustrative example: A common phrase can fetch a vector, but context still determines its meaning here.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Implementation details
Should Engram replace h_t with retrieved m_t?
Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models
Show answer and example
Answer: No. h′=h+sigmoid(W[h;m])⊙m preserves the residual and gates by context. Record bucket count, n-gram orders, and added-parameter training budget; preserve small-budget negative results.
Illustrative example: A poorly trained memory should not overwrite the original hidden state unconditionally.
Sections in the Chinese source: 核心公式, 本地复现. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Method comparison
How do Engram and MoE increase different resources?
Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models
Show answer and example
Answer: MoE routes compute experts. Engram reads embeddings by n-gram address. O(1) lookup still requires storing, training, and moving table values.
Illustrative example: Neural expert execution differs from fetching a phrase vector.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Basics
Why add lookup memory for frequent local patterns?
Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models
Show answer and example
Answer: Reconstructing common patterns repeatedly consumes compute. A learned n-gram table provides gated cached representations so attention and FFNs can focus on combinations.
Illustrative example: Look up a common phrase locally and compute novel combinations in the backbone.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Core mechanism
Why build hard negatives from search, browsing, and booking journeys?
Applying Embedding-Based Retrieval to Airbnb Search
Show answer and example
Answer: Random listings often differ obviously by geography or conditions. Browsed but unbooked listings are closer decision alternatives, though not necessarily disliked; explain sampling and retain serving filters/versioning.
Illustrative example: Within a Paris trip, similar local listings are harder than Tokyo, but sold-out homes are not necessarily irrelevant.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Implementation details
How match hard negatives and eligible candidates with targets?
Applying Embedding-Based Retrieval to Airbnb Search
Show answer and example
Answer: Use a positive from the same request and reasonable unselected journey listings in the contrastive denominator. Keep serving versions consistent. Local movie proxies lack inventory/geography.
Illustrative example: Two viewed but unbooked nearby homes are harder than an unavailable home in another city.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Method comparison
How does traffic replay differ from static negatives?
Applying Embedding-Based Retrieval to Airbnb Search
Show answer and example
Answer: Replay reconstructs real requests, journey candidates, and filters; static negatives may be too easy. Replay better matches first-stage competition but not live feedback or changing inventory.
Illustrative example: Distinguish New York listings available on matching dates, not just New York versus nationwide random homes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Basics
Why combine query/listing towers with geography and inventory filters?
Applying Embedding-Based Retrieval to Airbnb Search
Show answer and example
Answer: Dot-product retrieval finds relevant homes, but availability and destination remain request-specific constraints. An index accelerates matching without making eligibility inferable from similarity.
Illustrative example: A highly relevant apartment can be in the wrong city or unavailable.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why reward intermediate SID paths in PROMISE rather than score only complete items?
PROMISE: Process Reward Models Unlock Test-Time Scaling Laws in Generative Recommendations
Show answer and example
Answer: Early wrong semantic branches waste later computation. A lightweight process reward supplies dense training feedback and prunes drifting beams at inference. Wider beams can explore better paths but depend on reward quality and cost, not automatic improvement.
Illustrative example: Illustration: Reject a home-decoration prefix early for a hiking request instead of waiting to generate a cushion.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Why define comparable scales across prefix lengths in PROMISE reward accumulation?
PROMISE: Process Reward Models Unlock Test-Time Scaling Laws in Generative Recommendations
Show answer and example
Answer: Long paths naturally sum more terms, potentially creating length bias. Compare beam candidates at consistent depths or with defined length handling, calibrate log-probability and λR, and retain legality, deduplication, and termination separately.
Illustrative example: Illustration: Three positive rewards are not inherently better than one if the paths have different depths. High PRM cannot legitimize an illegal item.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
What extra judgment does PRM-guided beam widening add beyond likelihood?
PROMISE: Process Reward Models Unlock Test-Time Scaling Laws in Generative Recommendations
Show answer and example
Answer: Likelihood prioritizes current model probability; PRM adds process value, allowing lower-likelihood but task-relevant paths. Scoring costs compute and can be biased. Test-time scaling is a budget–quality curve, not a guarantee.
Illustrative example: Illustration: PRM can keep a less popular but correct branch; a popularity-biased PRM may still explore the wrong region.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
What does a SID prefix represent in PROMISE, and why does pruning remove many items?
PROMISE: Process Reward Models Unlock Test-Time Scaling Laws in Generative Recommendations
Show answer and example
Answer: It is a hierarchical address prefix shared by a reachable subset of items. Descendants only refine that branch. Pruning can remove many unexplored items at once, so a bad process score can cause broad misses.
Illustrative example: Illustration: Pruning photography excludes all cameras, lenses, and accessories below it, including a perfectly relevant lens.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why align task ranks before fusing raw predicted probabilities in HarmonRank?
Show answer and example
Answer: Different base rates and scales make raw fusion sensitive to calibration and sparsity. Differentiable rank-sum learns within-task order; inter-task attention models dependencies such as purchases and comments before unified ranking. Rank alignment discards magnitude and is not a general probability calibrator.
Illustrative example: Purchase probabilities are much smaller than click probabilities; ranking candidates within each task reduces raw-scale dominance.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Why require both classes and identify invalid tasks in a soft-AUC batch?
Show answer and example
Answer: AUC needs comparable positive-negative pairs; one-class slices must be masked or reported not estimable, not scored as perfect. Align candidate IDs across tasks. Temperature and sampling affect the differentiable approximation and its gradients, not actual full-data AUC.
Illustrative example: An all-zero purchase batch has no pairs and must not send a perfect score into task fusion.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
How does inter-objective self-attention handle conflicts differently from fixed fusion?
Show answer and example
Answer: Fixed coefficients impose a global trade-off. Attention learns dependencies and conditional combinations, increasing flexibility but risking domination by frequent objectives. Check by task and example; the local negative result does not establish quality benefits.
Illustrative example: A highly clicked but rarely purchased item can be treated differently from one high on both, instead of applying the same 7:3 ratio.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why move from classification scores to rank relationships in HarmonRank?
Show answer and example
Answer: Different target base rates and scales can dominate arithmetic fusion. Differentiable rank-sum/AUC first emphasizes each task's internal order, then learns task dependencies. Ranking loses absolute value magnitude rather than preserving every business unit.
Illustrative example: A purchase score can be numerically tiny but still rank items meaningfully, without saying how much more value one brings.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Core mechanism
How does Podcast MTL use organic promotion for cold ads, and why balance gradients?
Cold-Starting Podcast Ads and Promotions with Multi-Task Learning on Spotify
Show answer and example
Answer: User, content, and context representations share abundant organic signals while separate heads preserve objectives. Sample and gradient imbalance can drown the sparse advertising task, so adjust contributions. Transfer can fail when tasks differ; sharing does not guarantee gains.
Illustrative example: Illustration: A new podcast has organic plays but no ad clicks. Learn audience clues in the shared encoder while preventing organic gradients from overwhelming ad supervision.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Implementation details
Why not fill missing Podcast MTL labels with zeros?
Cold-Starting Podcast Ads and Promotions with Multi-Task Learning on Spotify
Show answer and example
Answer: An organic example may never expose the ad, so its ad response is unobserved, not negative. Mask that loss; observed objectives can still train shared features. Record task identity, data construction, and α. Movies proxy for podcasts only.
Illustrative example: Illustration: Listening after organic exposure does not imply the user ignored an unseen ad.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Method comparison
How does gradient balancing treat a small podcast task differently from mixing by counts?
Cold-Starting Podcast Ads and Promotions with Multi-Task Learning on Spotify
Show answer and example
Answer: Counts let the large task dominate shared weights. Balancing actual contributions preserves the sparse ad direction, but too much can amplify noise. Evaluate per task; duplicating records cannot guarantee cold-start gains.
Illustrative example: Illustration: One million organic and one thousand ad examples can bury ad gradients. Moderate balancing does not mean infinitely upweighting a single ad label.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Basics
Does cold start in Podcast MTL mean no content or no target feedback?
Cold-Starting Podcast Ads and Promotions with Multi-Task Learning on Spotify
Show answer and example
Answer: The gap is ad click/play labels for new podcasts, not necessarily missing titles or topics. Shared content and context help without representing observed ad outcomes. Transfer supplies a training signal, not a real click log.
Illustrative example: Illustration: A show's host and description can suggest likely audiences before exposure, but cannot verify their ad clicks.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why does HiGR need whole-page preference alignment after choosing semantic clusters then items?
HiGR: Industrial-Scale Hierarchical Generative Slate Recommendation Framework in Tencent
Show answer and example
Answer: Hierarchical codes make identification easier but individual relevance does not ensure useful order or complementarity. The Hierarchical Slate Decoder generates coarse-to-fine choices; ORPO reshapes combinations from preferred and rejected pages. Evaluated preferences do not guarantee long-tail or utility gains.
Illustrative example: Sports clusters can yield only near-duplicate shoes; page preferences can add socks or training content.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Why define HiGR sequence probabilities and numerical boundaries before calculating log odds?
HiGR: Industrial-Scale Hierarchical Generative Slate Recommendation Framework in Tencent
Show answer and example
Answer: Page probability is a product of conditional token probabilities and can underflow; use stable log-space arithmetic for both the joint and its complement. Arbitrary scores are not probabilities. Match hierarchical mappings and repetition constraints to page identity.
Illustrative example: Taking logs after a long-page probability rounded to zero fails; silently replacing the sequence probability with a length average changes the objective.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
How does HiGR ORPO preference evidence differ from page-level SFT?
HiGR: Industrial-Scale Hierarchical Generative Slate Recommendation Framework in Tencent
Show answer and example
Answer: SFT imitates positive pages. ORPO compares preferred and rejected page odds to teach relative composition. Preference and quantization biases remain, and local NDCG declined; report it with head-share changes instead of declaring overall improvement.
Illustrative example: An equally relevant but more complementary page can beat a repetitive one; a winner-only target provides no explicit relative signal.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why choose semantic clusters before items in HiGR's page generation?
HiGR: Industrial-Scale Hierarchical Generative Slate Recommendation Framework in Tencent
Show answer and example
Answer: Hierarchy divides a large space into coarse interests and precise identities, enabling composition across groups. Wrong coarse codes restrict later candidates, and apparent diversity does not prove relevance. Evaluate both quantization and whole-page utility.
Illustrative example: Choose tutorial, tool, and example groups before specific items, while recognizing a wrong entertainment branch can remain diverse but irrelevant.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Core mechanism
Why constrain mHC residual mixing to a doubly stochastic matrix?
mHC: Manifold-Constrained Hyper-Connections
Show answer and example
Answer: Repeated unconstrained mixing can amplify deep residual signals. Nonnegative entries and unit row/column sums constrain exchanges while retaining communication. This does not bound growth of the whole nonlinear network.
Illustrative example: Two streams can mix 0.7/0.3 under both constraints; repeated unconstrained sums may grow quickly.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Implementation details
Are unit row sums enough to validate mHC?
mHC: Manifold-Constrained Hyper-Connections
Show answer and example
Answer: No. Check nonnegativity and unit column sums as well as separate pre/post constraints. A nonexpansive residual mixer does not prove the full nonlinear model is nonexpansive or better trained.
Illustrative example: Two rows concentrating on one column satisfy row sums but fail column sums.
Sections in the Chinese source: 核心公式, 本地复现. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Method comparison
Why is mHC more than another normalization layer?
mHC: Manifold-Constrained Hyper-Connections
Show answer and example
Answer: It restricts residual maps to nonnegative doubly stochastic matrices and pre/post maps to nonnegative values. Hidden normalization does not impose those operator constraints.
Illustrative example: Normalized outputs do not imply row and column sums of one.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Basics
What freedom do multiple residual streams add?
mHC: Manifold-Constrained Hyper-Connections
Show answer and example
Answer: They exchange information dynamically, build attention/FFN inputs, and write back to several streams. Mixing needs constraints because it can amplify depth.
Illustrative example: Keep different histories and mix as needed rather than using one cumulative vector.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why revise or retire skills from successful trajectories after later failures?
Show answer and example
Answer: Success establishes utility under past conditions, not new ones. Returns train selection and reveal obsolete content. Revision and eviction stop accumulation of errors. An append-only library can keep retrieving relevant-looking stale skills and reinforce incorrect behavior.
Illustrative example: Example: A download skill worked before a site redesign. Update or disable it rather than rank it first based on old success.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
Can SAGE and sage-planner share implementation conclusions because they have the same name?
Show answer and example
Answer: No. SAGE concerns an RL-driven skill library; sage-planner concerns symbolic action gates and suffix edits. The local SAGE run tests skill state and budgets, without paper-scale models or benchmarks. Shared names do not fill evidence gaps.
Illustrative example: Example: A skill-library revision invokes no symbolic gate. Gate tests from the other method cannot verify this one.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
How does learned retrieval qφ differ from fixed semantic nearest neighbors?
Show answer and example
Answer: Semantic nearest neighbors use description similarity. Learned retrieval adapts to which skills improve execution returns. Outcomes are confounded by environment and task difficulty, so success after invocation is not independent causal contribution. Inspect updates with bank state and visible evidence.
Illustrative example: Example: Two skills sound like file organization. A more reliable route can get a higher learned score even if it shares fewer words.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Why make use of accumulated skills a learning objective in SAGE?
Show answer and example
Answer: Stored skills need timely, correct selection and can harm when mismatched. SAGE connects experience-derived skills to retrieval and uses later outcomes to govern their lifecycle. It studies durable use of experience, not library size or successful trace count.
Illustrative example: Example: A migration skill is irrelevant to looking up one row. Knowing when not to invoke it is useful skill selection.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why use hierarchical agents and dynamic meta-prompts rather than several fixed reasoning chains in RecGPT-V2?
Show answer and example
Answer: Fixed chains can repeat interests and ignore context. Hierarchical agents separate intents, compress behavior, and construct task-specific prompts so paths specialize. Rewards and judges help refine outputs, but agent count does not ensure independent knowledge; check repetition, bias, and cost.
Illustrative example: Illustration: Extract travel and photography needs separately instead of repeating 'likes photography' across three templates.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
How avoid counting duplicated RecGPT-V2 paths as extra ability in π=Σgkπk?
Show answer and example
Answer: Align path distributions over one candidate space, normalize the gate across actual paths, and check duplicated outputs and call costs. Copies of one policy add no information. Fixed-score proxies do not reproduce hierarchical memory and dynamic prompts merely by mixing.
Illustrative example: Illustration: Three paths returning the same ten popular items do not triple interest coverage.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
How do agent judges and real user feedback differ in evaluation bias?
Show answer and example
Answer: Judges quickly score semantics or format; users reveal behavior, but exposure biases it. Mixed rewards can use both without treating either as true satisfaction. Self-proposal and self-judging can reinforce bias, requiring independent validation. Local proxies do not establish production judging/RL.
Illustrative example: Illustration: A judge may like long explanations while users want quick results. Record semantic scores separately from behavior.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Which RecGPT-V2 inputs are evidence and which are reasoning instructions?
Show answer and example
Answer: Compressed behavior summarizes observations; the meta-prompt specifies how to use them. Instructions can shift attention but cannot create facts or recover lost evidence reliably. Preserve this boundary across agents.
Illustrative example: Illustration: Use 'prioritize discovery' or 'prioritize practical choices' on the same history without inventing a diving trip.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
Why retain engine correction when Online IcePop updates only once?
Show answer and example
Answer: Fresh one-use rollouts eliminate reuse staleness, so policy ratio at sampling is one and PPO clipping unnecessary. Equal weights can still produce different probabilities in training vs inference; IcePop keeps its two-sided mask and in-range ratio.
Illustrative example: A fresh answer can come from a lower-precision engine; freshness is not engine equality.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
How verify Online IcePop never silently reuses a batch?
Show answer and example
Answer: Track unique batch IDs and consumption; reject a second update. A local candidate group is only a scheduling proxy, not distributed rollout infrastructure.
Illustrative example: Retried delivery of the same batch is not fresh data.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
Which correction does Online IcePop remove?
Show answer and example
Answer: Remove reuse-induced stale-policy ratio and PPO clip, not the two-sided engine mask or original in-range ratio. Repeated updates invalidate the online simplification.
Illustrative example: Scheduling must enforce one use; deleting a line doesn't make training online.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
Why consume each Online IcePop rollout batch once?
Show answer and example
Answer: One update avoids extra staleness from reuse, making initial policy ratio one. Train/inference numerical correction remains.
Illustrative example: Discard the fresh batch after its update instead of optimizing against it again.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
Why split engine mismatch and policy staleness, and where does routing replay help?
Show answer and example
Answer: Numerical engine differences and weight drift are distinct. Correct on-policy probabilities, combine off-policy updates with clipping and replay rollout MoE routing to reduce path mismatch. Both probabilities and paths need testing; local kernels aren't a cluster.
Illustrative example: One token may route to different experts even with synchronized weights, which a stale-policy ratio does not fix.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
Does the local miniRL kernel prove distributed MoE stability?
Show answer and example
Answer: No. It checks correction, clipping and routing state semantics. Real replay needs recorded token routes and actual aligned execution, not a label.
Illustrative example: A replay flag doesn't prove forward passes used the recorded experts.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
What do MoE Routing Replay and ratio clipping address?
Show answer and example
Answer: Replay restores expert execution paths; clipping limits update magnitude from probabilities. Clipping cannot reconstruct old routes.
Illustrative example: Saving which expert executed is different from lowering a large ratio.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
Why model both train/inference mismatch and staleness in miniRL?
Show answer and example
Answer: Equal weights may score differently across engines; old/new weights drift through learning. Separate them for appropriate on/off-policy correction.
Illustrative example: A kernel change and a policy update are distinct causes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
Why must task verifiers check majority votes rather than treating agreement as truth?
Show answer and example
Answer: Executors may share wrong assumptions. Voting measures agreement, not independent factual correctness. External task rules verify candidates, and curricula advance near the current ability boundary. If the generator or voter sees references, leakage invalidates claims of autonomous tool capability.
Illustrative example: Five executors can agree on the same wrong unit conversion. A verifier uses the task contract, not the number of votes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
Can Agent0's verifier return its internal solution or gold answer directly to executors?
Show answer and example
Answer: Execution and evaluation inputs should contain public task conditions only; hidden references stay with the verifier. Check task executability and feedback provenance. Saturated mini-suite success cannot establish LLM self-evolution.
Illustrative example: If the generator hides a canonical action sequence in the execution prompt, success no longer establishes autonomous solving.
Sections in the Chinese source: 核心公式, 复现边界. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
Do agent majority agreement and independent verifier acceptance provide the same evidence?
Show answer and example
Answer: Agreement checks whether candidates match; verification checks task satisfaction. Shared errors can win a vote. Candidate groups cannot prove themselves correct, and agreement alone cannot establish improvement for curriculum advancement.
Illustrative example: Five identical wrong results still agree, but the verifier should mark failure.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why does Agent0 create a curriculum with a task generator and several execution agents?
Show answer and example
Answer: Verifiable tasks create training opportunities automatically, multiple candidates give group feedback, and difficulty rises with the capability frontier. Zero human data does not mean no success criteria; the verifier still constrains curriculum quality.
Illustrative example: After executors master a task family, propose a slightly harder but still verifiable task.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
Why do independent interaction transitions help switch between token-level and step-level credit?
Show answer and example
Answer: They explicitly preserve state, action, and next-observation boundaries, enabling attribution at generated-token or environment-step granularity instead of flattening the transcript. Pluggable context management cannot change transition semantics. Tool output stays an environmental observation, not a policy action loss term merely because it is logged together.
Illustrative example: A query can return a long table and still be one action. Assign step credit or split credit only among tokens that generated the query.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
How do Agent-R1 transition logs prevent the next observation from becoming visible to the current action?
Show answer and example
Answer: Keep temporal boundaries among s_t, a_t, and o_{t+1}; generating a_t cannot read o_{t+1}. Cut advantage recurrence at episode termination. Local mechanism updates are not actual multi-GPU agent training.
Illustrative example: Before a tool call, policy input cannot already contain its future response.
Sections in the Chinese source: 核心公式, 复现边界. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
Are token-level and step-level credit just two loss names?
Show answer and example
Answer: No. Decision units, value boundaries, and action-length treatment differ. Explicit transitions make choices auditable. Renaming the optimizer does not fix confused observation and policy-response boundaries.
Illustrative example: A tool action with 20 tokens is one environment step, not 20 pieces of environment feedback.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why does Agent-R1 record each agent–environment interaction separately?
Show answer and example
Answer: A transition links state, action, and next observation for pluggable context management and environments, plus selectable token- or step-level credit. It is more structured than one text example containing the whole chat.
Illustrative example: A tool call and its return form a distinct training record.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why route long- and short-term interests separately in DualGR instead of concatenating history?
DualGR: Generative Retrieval with Long and Short-Term Interests Modeling
Show answer and example
Answer: Recent and stable interests matter differently by request. Two paths learn generation distributions, a gate mixes them conditionally, valid prefixes map outputs to items, and exposure terms limit dominance of popular codes. Check distinct learning and gate collapse.
Illustrative example: A long travel history should not drown out today's repeated kitchen-shopping actions.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Why align item columns before mixing the two DualGR distributions?
DualGR: Generative Retrieval with Long and Short-Term Interests Modeling
Show answer and example
Answer: Gate-weighted addition needs probabilities for the same item IDs. Align codebooks and candidates, map legal prefixes to serving items, and derive exposure only from training-available data. Local proxies do not cover production routing.
Illustrative example: If column one means camera on one path and children's clothes on the other, adding the values is semantically invalid despite matching shapes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
What freedom does DualGR's two-path routing add over fixed exponential decay?
DualGR: Generative Retrieval with Long and Short-Term Interests Modeling
Show answer and example
Answer: Fixed decay prescribes how age changes influence. Separate learned statistics and request-level gating can adapt nonlinearly, with more parameters and calibration. Recent noise or stale long-term interests can still dominate.
Illustrative example: One accidental click and a burst of consistent demand can receive different gate responses that are hard to express with one decay rate.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Why does DualGR keep separate short- and long-term generation distributions?
DualGR: Generative Retrieval with Long and Short-Term Interests Modeling
Show answer and example
Answer: A single compressed representation can let more numerous historical events dominate current needs. Separate learned paths are dynamically mixed with valid-prefix and exposure constraints, not simply duplicated vectors or a guarantee of detecting every new interest.
Illustrative example: Keep established photography interest and new baby-product demand distinct enough to adjust their weights.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why place behavior before non-sequence fields in OneTrans and use different parameter sharing for each?
Show answer and example
Answer: Causal order lets later fields read behavior while keeping history independent of later candidates. Homogeneous events share QKV and FFN, while heterogeneous fields use position-specific parameters. Pyramid shortening saves later query work. Reuse still depends on time window, versions, and dependencies.
Illustrative example: Illustration: Reuse earlier click history when swapping a candidate; candidate fields read that history using parameters suited to price and category.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why do OneTrans NS order and causal masking determine visibility?
Show answer and example
Answer: S-first lets later fields read history while history cannot read later candidates, enabling shared state. Visibility between NS positions depends on the actual mask. Validate history, time encoding, and model versions before reusing cross-request caches, not just user ID.
Illustrative example: Illustration: Move a candidate before history and later events may see it, invalidating the old cache contract.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
Where does OneTrans's pyramid of S queries save work compared with keeping full length in every layer?
Show answer and example
Answer: Later layers update fewer recent queries, reducing output and interactions. Older evidence can survive through earlier states or legal K/V; it is not dropped from the input or absent everywhere. The small-data head concentration is not generalization evidence.
Illustrative example: Illustration: Read broad history in shallow layers and focus on recent queries later, unlike deleting distant events first.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why share S-token weights but use position-specific NS-token weights in OneTrans?
Show answer and example
Answer: Behavior events have similar roles, whereas user and item fields are heterogeneous. Joint causal interaction does not make them homogeneous. Position-specific weights cost capacity and must move with field order.
Illustrative example: Illustration: Clicks share QKV, but age and price may use separate FFNs. Swapping fields without their mapping treats price like age.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
How is zeroing out-of-range IcePop updates different from clipping to a boundary?
Show answer and example
Answer: Clipping preserves boundary-weighted contribution; masking drops the current token gradient. IcePop uses two fixed bounds and original in-range correction. It filters mismatch, not text quality.
Illustrative example: Tiny ratio is unaffected by upper clipping, but two-sided masking excludes it instead of raising it to the lower bound.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
Must IcePop distinguish engine ratio from policy ratio?
Show answer and example
Answer: Yes. Engine ratio measures implementation mismatch; policy ratio measures current vs sampling policy. Mask the specified engine ratio only. No local real MoE/asynchronous system.
Illustrative example: Using model-learning drift as the engine mask could drop normal probability changes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
How do IcePop masking and TIS clipping treat out-of-range samples?
Show answer and example
Answer: IcePop zeroes their gradient; TIS caps high weights but keeps them contributing. IcePop uses original in-range ratios, not ones.
Illustrative example: An oversized ratio can retain capped contribution in TIS but none in IcePop.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
Why does IcePop mask very small train-to-serve ratios?
Show answer and example
Answer: Upper clipping only controls huge weights; tiny ratios may also signal mismatch. A two-sided interval keeps trustworthy ratios without snapping others to boundaries.
Illustrative example: A training probability far below serving can be unreliable even though its weight is small.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Core mechanism
Why does CRSD compare reasoning and no-reasoning views in one student instead of serving reasoning chains?
From Reasoning LLMs to BERT: A Two-Stage Distillation Framework for Search Relevance
Show answer and example
Answer: The reasoning view offers richer relationships; the plain view is the low-latency input used in production. Self-distillation and contrastive learning transfer judgment to the plain path while label supervision anchors correctness. Leaked answers or view distribution mismatch can make a lower loss misleading.
Illustrative example: Train relevance with an attribute-comparison rationale, then require similar decisions from query and item alone without generating a rationale per candidate.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Implementation details
Why would appending reasoning r at serving violate CRSD's distillation objective?
From Reasoning LLMs to BERT: A Two-Stage Distillation Framework for Search Relevance
Show answer and example
Answer: The plain `pS(·|x)` learns from augmented `pS(·|x,r)` specifically to remove online rationale generation. Requiring r restores cost and does not verify plain-path learning. Test training losses and rationale-free inference separately; local proxies do not verify full domain LLM training.
Illustrative example: Generating rationales before scoring tests the augmented pipeline; serving only raw x tests the intended lightweight path.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Method comparison
How does CRSD's contrastive self-distillation differ from fitting external teacher logits?
From Reasoning LLMs to BERT: A Two-Stage Distillation Framework for Search Relevance
Show answer and example
Answer: One lightweight student produces distributions under both views, and a contrastive loss also constrains representations. Shared parameters can reduce scale mismatch but do not eliminate erroneous rationales. Check actual gradient and stop-gradient paths rather than assuming self-distillation is unbiased.
Illustrative example: The same model reads ordinary product information and an augmented rationale; a separate fixed teacher would not share its parameters.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Basics
Why is reasoning a training augmentation in CRSD rather than a permanent serving requirement?
From Reasoning LLMs to BERT: A Two-Stage Distillation Framework for Search Relevance
Show answer and example
Answer: Rationales can enrich supervision but cost real-time generation. A student learns from plain and rationale-augmented views, transferring augmented-view distributions to the plain path. This does not guarantee retention of complete reasoning or faithful explanations.
Illustrative example: Train with a commuting-lightweight rationale, but serve on current features without first generating a long explanation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why use hierarchical video SIDs and continue language pretraining in PLUM instead of enlarging an ID table?
PLUM: Adapting Pre-trained Language Models for Industrial-scale Generative Recommendations
Show answer and example
Answer: Huge ID tables memorize items and share little with new ones. Composable SIDs and CPT connect behavior, text, ASR, and metadata to LM capacity before next-SID training. Beam search generates candidates online, but code quality still determines retrieval usefulness.
Illustrative example: Illustration: A new baking video can borrow a coarse baking code and distinguish itself at a fine level instead of learning an isolated ID from scarce clicks.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Why reconstruct the first r codeword layers in PLUM progressive masking instead of shuffling levels?
PLUM: Adapting Pre-trained Language Models for Industrial-scale Generative Recommendations
Show answer and example
Answer: Residual levels accumulate coarse-to-fine meaning. Random prefix depths require each prefix to work; shuffling destroys that order. Mask SFT prompt loss, supervise only target SIDs, and use catalog trie legality. Training on prompt text changes the objective.
Illustrative example: Illustration: Sometimes reconstruct with coarse codes and sometimes add detail; using level three as the first code is not progressive reconstruction.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
Why cross random versus LM initialization with CPT on versus off in PLUM?
PLUM: Adapting Pre-trained Language Models for Industrial-scale Generative Recommendations
Show answer and example
Answer: Four cells separate initialization, extra continued training, and their interaction. Locally CPT lowered loss without improving Recall and LM-initialized arms had zero hits. Convergence is not successful retrieval or general scaling evidence.
Illustrative example: Illustration: Run random/no CPT, random/CPT, LM/no CPT, and LM/CPT to distinguish pretrained knowledge from extra training.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Why mix behavior and SID–metadata examples during PLUM CPT instead of only tuning next-item codes?
PLUM: Adapting Pre-trained Language Models for Industrial-scale Generative Recommendations
Show answer and example
Answer: Behavior teaches consumption transitions; text and ASR connect addresses to language semantics. CPT adapts pretrained knowledge to recommendation, then SFT focuses on retrieval. Mixture ratio matters, and lower loss need not improve search.
Illustrative example: Illustration: Learn both that a code means a camping tutorial and what users consume afterward.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Core mechanism
Why learn inverse dynamics from tutorials instead of treating before/after pixel differences as mouse actions?
Watch and Learn: Learning to Use Computers from Online Videos
Show answer and example
Answer: Screen changes may come from scrolling, keystrokes, or program responses, so pixels do not identify actions uniquely. The inverse model predicts action type, coordinates, and text from screen pairs, then assembles trajectories. These are estimated labels, not rule guesses or proof an IDM was trained locally.
Illustrative example: Example: A screen can move down from a wheel or a key. Use visual context and supervision instead of labeling every downward shift as a click.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Implementation details
Why can't one simple rule head replace action, coordinate, and text prediction?
Watch and Learn: Learning to Use Computers from Online Videos
Show answer and example
Answer: The paper uses action classification, coordinates discretized into 0–1000 classes, and a GPT-2 small autoregressive text head with SigLIP-2 and a four-layer Transformer. Do not invent loss weights. Local work is literature and cost review, not IDM training, corpus download, or desktop evaluation.
Illustrative example: Example: A cursor-difference click rule cannot recover typed text from two frames or replace joint IDM training.
Sections in the Chinese source: 核心公式, 本地复现. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Method comparison
What changes when retrieving video examples versus training on annotated trajectories?
Watch and Learn: Learning to Use Computers from Online Videos
Show answer and example
Answer: Retrieval puts a relevant demonstration in context without updating the executor. SFT writes trajectory patterns into weights. Both rely on IDM label quality, so video count is not correct action count. Similar titles do not establish actual desktop learning.
Illustrative example: Example: Use one tutorial as a prompt example or offline training input. Inference cost and how errors propagate differ.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Basics
Why use online tutorial videos rather than rely only on expensive action annotations?
Watch and Learn: Learning to Use Computers from Online Videos
Show answer and example
Answer: Tutorials contain genuine software behavior without machine-readable click coordinates, keys, and actions. Converting visible demonstrations broadens training sources. Frames are not action ground truth: occlusion, edits, and missing typed text can harm conversion and execution.
Illustrative example: Example: A file moves across windows, but training data needs to distinguish dragging from a copy command rather than invent a click.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why separate orchestration and execution memory for recombination on new tasks?
Show answer and example
Answer: Decomposition and delegation capture high-level dependencies, while action templates ground calls. One full trace binds them to particular people and environments. Layers can reuse structure while swapping execution units. Admit verified units only and recheck interfaces and prerequisites when combining them.
Illustrative example: Example: Collect, verify, and summarize stays the same while data comes from a database in one task and a file in another.
Sections in the Chinese source: 背景与主要改动, 核心公式与算法. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
Does local reused_plans verify real multi-agent Office delegation?
Show answer and example
Answer: No. The local run extracts action/domain units from structured verifiable plans and reuses high-level and fine-grained steps. It does not execute Office apps, LLM delegation, or complete recovery. Reuse counts are neither OfficeBench success nor real tool cost.
Illustrative example: Example: A rule reuses a subplan 100 times without showing a model can delegate between real document and email apps.
Sections in the Chinese source: 核心公式与算法, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
What generalization trade-off separates procedural recombination from whole-trace retrieval?
Show answer and example
Answer: Whole traces preserve complete context but may be tied to one task. Procedural units recombine for new needs but can lose cross-unit prerequisites. Legal dependencies and validation are needed; individual success does not imply arbitrary compositions work.
Illustrative example: Example: Reading a spreadsheet and sending email may both work alone. Verify recipients and data permissions before combining.
Sections in the Chinese source: 背景与主要改动, 核心公式与算法. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Why does LegoMem focus on recurring workflows with multiple roles and steps?
Show answer and example
Answer: Office workflows share portions but change goals and roles. Reorganizable procedures let new tasks reuse the right steps rather than copy one complete run. Units must retain dependencies and context; pieces that look composable can still lack inputs.
Illustrative example: Example: Weekly reports and project briefs both need collection and source verification, but different approvers. Reuse collection and replan handoff.
Sections in the Chinese source: 背景与主要改动, 核心公式与算法. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Core mechanism
Why can testing all cyclic phases still miss full-period deletion?
Detecting Post-generation Edits to Watermarked LLM Outputs via Combinatorial Watermarking
Show answer and example
Answer: Phase search handles global offsets, but deleting a whole period can preserve the repeating pattern and phase. Independently calibrate global/local thresholds; a hard-pattern diagnostic is not perfect soft-watermark localization. No full local model edit matrix.
Illustrative example: Removing one full red/blue cycle leaves the remaining pattern unchanged.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Implementation details
Can local E(t) zero-pad edges?
Detecting Post-generation Edits to Watermarked LLM Outputs via Combinatorial Watermarking
Show answer and example
Answer: No. It is defined only for complete interior t=w−1 through T−w, without zero padding or changing denominators. Check all phases and preserve full-period deletion as a blind negative control.
Illustrative example: Edge positions with incomplete windows should not be falsely flagged because zeros were added.
Sections in the Chinese source: 核心公式, 本地复现. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Method comparison
Can global D and local E(t) thresholds replace each other?
Detecting Post-generation Edits to Watermarked LLM Outputs via Combinatorial Watermarking
Show answer and example
Answer: D detects a watermark across the whole passage; E(t) uses window coverage to flag possible edits. They differ in scope and threshold direction. Calibrate false positives independently.
Illustrative example: Global passage can pass while a short span scores low; that does not precisely identify every edit.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Basics
Why bias vocabulary partitions by a periodic pattern?
Detecting Post-generation Edits to Watermarked LLM Outputs via Combinatorial Watermarking
Show answer and example
Answer: Tag-specific logit biases induce a statistical periodic structure. The detector compares windows across all cyclic shifts for watermark and edit evidence.
Illustrative example: AB partition alternation is detectable without knowing where the observed text starts.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Core mechanism
Why can't Autorater backpropagate through parsed probability text?
Judging with Confidence: Calibrating Autoraters to Preference Distributions
Show answer and example
Answer: The policy generates text, parses it to a fixed reward and uses policy gradients. Discrete text-to-float parsing has no ordinary differentiable path. Brier/log rewards encourage calibration, with separate invalid penalties. Derivatives of numeric scores alone are not GRPO.
Illustrative example: An unformatted 'very confident' response fails parsing; don't silently substitute a plausible number.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Implementation details
Why can't invalid numbers enter the formula before multiplying by a zero mask?
Judging with Confidence: Calibrating Autoraters to Preference Distributions
Show answer and example
Answer: Separate invalid outputs first. NaN can propagate through nonlinearities and multiplication by zero; out-of-range values are not probabilities. Use the parse-failure reward. The local strict parser is a contract, not the authors' parser or full RL.
Illustrative example: Route NaN directly to invalid reward rather than computing log(NaN) first.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Method comparison
How do Brier and log rewards penalize confident errors differently?
Judging with Confidence: Calibrating Autoraters to Preference Distributions
Show answer and example
Answer: Both are proper probability scores, but squared and logarithmic tails differ; log loss is especially sensitive to extremely confident mistakes. Parse failures also have a separate reward, beyond legal probability gradients.
Illustrative example: A near-certain win prediction followed by defeat is penalized by both, with different magnitudes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Basics
Why should a probabilistic autorater output a preference probability, not just a winner?
Judging with Confidence: Calibrating Autoraters to Preference Distributions
Show answer and example
Answer: Real preferences can disagree. Brier/log rewards fit observed preference frequencies rather than force binary certainty. Reliable target distributions matter; arbitrary numbers are not calibrated.
Illustrative example: Near-even annotator votes warrant an intermediate probability rather than an absolute winner.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Core mechanism
Why rank each segment and exclude self from the peer mean?
When LLMs Benchmark Themselves: Deconstructing Self-Bias in Automated Evaluation
Show answer and example
Answer: Segment scales may differ, so rank before averaging to preserve the protocol. Remove self from the peer comparison. Score direction, axes and tie handling affect the sign; peer consensus is not external truth.
Illustrative example: A low-is-better metric and high-is-better metric need consistent rank direction. Never put the self score back into peers.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Implementation details
Why do axis order and leave-self-out matter for self-bias?
When LLMs Benchmark Themselves: Deconstructing Self-Bias in Automated Evaluation
Show answer and example
Answer: Rank within segments, average over segments, then subtract the peer mean excluding self from the diagonal. Specify whether scores are higher- or lower-is-better; negative bias indicates favorable self-rank.
Illustrative example: MetricX-QE is lower-is-better. Descending ranks would reverse the bias direction.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Method comparison
How does the protocol distinguish self-evaluation advantage from a stronger model?
When LLMs Benchmark Themselves: Deconstructing Self-Bias in Automated Evaluation
Show answer and example
Answer: Compare one target's ranks under its own role and peers to isolate evaluator preference from target quality. Peers can share biases, so this is relative audit evidence, not a definitive capability verdict.
Illustrative example: Consistently leading under all judges differs from leading only under one's own judge.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Basics
Which axis is target, which is benchmark role, and which cell measures self-evaluation?
When LLMs Benchmark Themselves: Deconstructing Self-Bias in Automated Evaluation
Show answer and example
Answer: The target is evaluated; the benchmark role performs evaluation. Self-evaluation is the diagonal where they match. Compare own and other roles for one fixed target, not different targets.
Illustrative example: A answering with B judging is off-diagonal; A answering and judging is diagonal. Keep A as the target in both.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
How does IntSR handle both explicit search terms and recommendation intent without a query?
IntSR: An Integrated Generative Framework for Search and Recommendation
Show answer and example
Answer: An explicit query expresses the current request; implicit session history supplies unspoken needs. Both inform latent intent, and results are generated in a unified item or POI vocabulary. Time context helps determine whether needs still apply and bridges search and recommendation. Sharing a vocabulary does not excuse ignoring query constraints or treating old interests as explicit current requests.
Illustrative example: Illustration: When a user searches for nearby coffee, a history of quiet venues can supplement the query. With no search, the session can suggest a need, but yesterday's location should not be assumed to apply today.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
How should IntSR distinguish an absent query from a real empty-string search?
IntSR: An Integrated Generative Framework for Search and Recommendation
Show answer and example
Answer: Use the missingness marker or branch specified by the model contract. Time conditioning may only use the request timestamp, and the target POI must not enter latent-intent estimation. Both entry points share item identity, not necessarily every label. The source's movie proxy cannot test real locations or opening hours.
Illustrative example: Illustration: Never starting a search and deliberately submitting a blank query can have different log meanings. Do not collapse them to one unexplained string and leave the model to guess.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
What does IntSR's latent-intent mixture make explicit compared with concatenating q and h for scoring?
IntSR: An Integrated Generative Framework for Search and Recommendation
Show answer and example
Answer: It first estimates an intent distribution and then generates items conditioned on intent and time, keeping multiple plausible needs explicit. Direct concatenation may express this implicitly without learning the same structure. A latent variable need not have a stable human meaning, and a more decomposable output does not prove recovery of true intent.
Illustrative example: Illustration: 'Weekend activities' could mean hiking or an exhibition. Keeping probabilities for both before choosing a venue is natural, but those probabilities remain model estimates.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Why combine explicit queries and implicit session intent in the same item space in IntSR?
IntSR: An Integrated Generative Framework for Search and Recommendation
Show answer and example
Answer: Search terms can omit background, while session activity offers context without directly stating the current request. Joint intent inference uses both, and a shared vocabulary avoids fragmented item identities across the two entry points. Time represents drift. An absent query should follow a defined missing-input path, not be replaced with something the user supposedly said.
Illustrative example: Illustration: A query of 'nearby' combined with a recent vegetarian search can suggest dining. A recommendation-only visit should use its session without inventing a user query.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
How do OnePiece's latent blocks and progressive task weights differ from simply summing all losses?
OnePiece: Bringing Context Engineering and Reasoning to Industrial Cascade Ranking
Show answer and example
Answer: Residual blocks repeatedly update contextual state before heads read it. Scheduled weights change which objectives dominate training stages so shared representations adapt to retrieval, click, and value. Both path and order matter; a local proxy does not prove verifiable linguistic reasoning in latent states.
Illustrative example: Illustration: Learn relevance before raising purchase-value weight to avoid an early sparse objective dominating basic ranking.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
How verify OnePiece's H(k+1) residual update and αt(k) schedule separately?
OnePiece: Bringing Context Engineering and Reasoning to Industrial Cascade Ranking
Show answer and example
Answer: One updates contextual state per block, the other weights losses per training stage. Distinguish the two meanings of k and verify both state propagation and applied loss weights. A fixed scorer proxy cannot establish full cascade training.
Illustrative example: Illustration: More blocks do not imply progressive weights, and changing losses does not establish multi-step latent updates.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does progressive weighting change learning order relative to fixed equal task weights?
OnePiece: Bringing Context Engineering and Reasoning to Industrial Cascade Ranking
Show answer and example
Answer: Fixed weights exert the same pressure throughout. A schedule lets basic relevance and harder value objectives converge in stages. This can reduce conflicts or cause forgetting, so inspect stage metrics. Proxy results do not make all curricula superior.
Illustrative example: Illustration: Emphasize relevance early and transaction value later; forgetting relevance could yield expensive but unrelated items.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why does OnePiece jointly cover retrieval, clicks, and value?
OnePiece: Bringing Context Engineering and Reasoning to Industrial Cascade Ranking
Show answer and example
Answer: Cascade stages have distinct labels and independent training can lose transferable clues. Shared context and latent updates model multiple needs, while progressive weights schedule learning. Objectives can conflict; a 'reasoning' name does not verify language reasoning.
Illustrative example: Illustration: Relatedness, click likelihood, and expected value complement one another, but high clicks need not imply purchases.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Does DRL-PUT learn which items to recommend or how to combine utility scores?
Deep Reinforcement Learning for Ranking Utility Tuning in the Ad Recommender System at Pinterest
Show answer and example
Answer: Actions specify weights for relevance, CTR, CVR, and related scores. A request-conditioned policy combines existing predictors, while offline updates correct action distribution differences using logging probabilities. Rarely logged weights lack evidence for claims of live optimality.
Illustrative example: Keep ad representations fixed but emphasize relevance for exploration and conversion for purchase intent.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Why must action identities match logged propensities in DRL-PUT's `π/μ`?
Deep Reinforcement Learning for Ranking Utility Tuning in the Ad Recommender System at Pinterest
Show answer and example
Answer: Keep executed weights, behavior probability, context, and reward tied to one event. With μ=0, no support exists; a tiny replacement does not enable unbiased extrapolation. Baseline b cannot read test outcomes, and final lists alone cannot reconstruct the update.
Illustrative example: Logging the propensity of 4:6 for an executed 7:3 mix gives the wrong gradient multiplier despite a genuine reward.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
How does DRL-PUT off-policy updating differ from fitting high-reward logged examples directly?
Deep Reinforcement Learning for Ranking Utility Tuning in the Ad Recommender System at Pinterest
Show answer and example
Answer: Logs are generated by μ and target choices follow π; the probability ratio addresses differing action frequencies. Without it, frequently tried actions dominate. Valid support and logging are required, extreme ratios increase variance, and public proxies are not randomized ad logs.
Illustrative example: If the old system nearly always emphasized CTR, most high rewards may follow that choice without showing alternatives are worse.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why make utility weights the action instead of generating items in DRL-PUT?
Deep Reinforcement Learning for Ranking Utility Tuning in the Ad Recommender System at Pinterest
Show answer and example
Answer: Existing CTR, CVR, relevance, and revenue models score candidates. The policy chooses a controlled mix, and feedback evaluates the induced list, not isolated causal item effects.
Illustrative example: A 7:2:1 versus 4:3:3 mix changes order while leaving item embeddings and the click model unchanged.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why supervise only the latest impression and share context K/V across OneRec-V2 decoder layers?
Show answer and example
Answer: Latest-impression supervision avoids full decoding for every history position. Context Processor produces one reusable K/V state and Lazy blocks read it to generate short SIDs. This reduces repeated history encoding and per-layer cache costs. Real-feedback alignment is separate and its effects cannot all be assigned to shared K/V.
Illustrative example: Illustration: For three SID levels after a long history, each layer reads one shared memory rather than building its own full cache.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Why compute DARS quantiles within a user and duration bucket and detach GBPO denominators?
Show answer and example
Answer: Buckets compare similar viewing opportunities instead of mixing user habits and lengths. Stop-gradient prevents the denominator from adding unintended derivatives. Separate explicit dislike negatives from other zero advantages. Labels stay in post-training, not request inputs.
Illustrative example: Illustration: Thirty seconds on a ten-second and ten-minute video are not directly comparable; differentiating through a numerically identical denominator changes the update.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
Why isn't GBPO's dynamic denominator equivalent to PPO clipping?
Show answer and example
Answer: PPO clips the probability ratio; GBPO uses different stopped-gradient denominators by advantage sign. These limit updates differently. Replacing them with clipping loses defining behavior. Local feedback-weighted probability is not NDCG or online lift, and not all three seeds improved.
Illustrative example: Illustration: Positive and negative advantages respond differently as current probability changes; a symmetric clip does not establish equivalence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Why use real playback and negative feedback instead of an extra reward model in OneRec-V2?
Show answer and example
Answer: Direct behavior avoids reward-model training and error propagation. DARS adjusts duration opportunity and GBPO controls signed updates. Real behavior is still a biased satisfaction proxy. Removing a reward model does not remove calibration or constraints.
Illustrative example: Illustration: Long videos offer more watch seconds. Compare in user and duration buckets instead of treating all long watches as high satisfaction.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Core mechanism
How does MPFormer's objective token produce retrieval representations for different goals from the same history?
MPFormer: Adaptive Framework for Industrial Multi-Task Personalized Sequential Retriever
Show answer and example
Answer: The objective token conditions the sequence encoder, so shared layers can extract different signals for different behavior goals. Dynamic weights and quotas adjust learning and retrieval budgets by difficulty. Calibrate goals separately; they are not interchangeable next-item targets.
Illustrative example: Illustration: Click retrieval can favor novelty while purchase retrieval emphasizes recent repeated comparisons, changing both features and candidate quota.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Implementation details
Why record how MPFormer's qm(h) is used for training weights versus candidate quotas?
MPFormer: Adaptive Framework for Industrial Multi-Task Personalized Sequential Retriever
Show answer and example
Answer: Normalized continuous weights multiply losses; serving counts need integer conversion and a total cap. Mask missing labels instead of making up zero supervision. Mixing proxy scores does not train an objective-conditioned shared model.
Illustrative example: Illustration: A training weight of 0.3 does not automatically mean 30 candidates; with total budget ten, define rounding and remainder.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Method comparison
How do MPFormer dynamic quotas change serving budgets relative to fixed K per task?
MPFormer: Adaptive Framework for Industrial Multi-Task Personalized Sequential Retriever
Show answer and example
Answer: Dynamic quotas allocate by difficulty or context, potentially increasing coverage for hard goals while crowding out others. Normalized weights do not ensure valid integer budgets. The public proxy does not establish gains for every industrial behavior objective.
Illustrative example: Illustration: Allocate less to easy click retrieval and more to sparse purchases, unless the difficulty estimate misallocates the extra capacity.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Basics
Why use objective tokens for MPFormer retrieval goals?
MPFormer: Adaptive Framework for Industrial Multi-Task Personalized Sequential Retriever
Show answer and example
Answer: Different feedback goals need different views of the same history. An objective token conditions a shared backbone, and task difficulty informs capacity and candidate allocation. A token is an interface, not supervision; renaming it cannot teach an unsupported behavior.
Illustrative example: Illustration: A user may click novel items and purchase trusted ones. Training only on views does not teach purchasing by swapping the objective token.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
Why exclude simulated-user messages from agent policy-token loss just like tool responses?
Show answer and example
Answer: They are produced by the environment's user policy or tools, not chosen by the agent. Supervising them confuses actions with conditions. Dynamic users support clarification and changing requirements, but final reward should measure completion rather than imitation of simulated user text. Simulation does not cover all real behavior.
Illustrative example: If a user changes the date to Friday, the agent should reschedule, not learn to generate the user's phrase 'move it to Friday' itself.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
Can the MUA-RL simulated user expose the scenario's complete hidden goal directly?
Show answer and example
Answer: Only reveal what a user could say under the interaction protocol, not private references or the entire solution. Loss covers agent responses only. A deterministic local user and mini tools are not an external LLM or production database.
Illustrative example: The user can reveal a constraint after clarification rather than attaching the full canonical tool sequence at the beginning.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
What differs between dynamic user simulation and tool results as the only environmental feedback?
Show answer and example
Answer: User messages can change or clarify goals; tool results mainly report post-action state. Both are environment inputs with different roles. Static tool traces cannot cover policies for evolving requests.
Illustrative example: A tool can report success and a user clarify that they wanted a different outcome; the agent still needs to understand the request.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why does MUA-RL include the user in dynamic rollouts instead of using only an initial request?
Show answer and example
Answer: Real users can clarify or change intent, requiring the policy to adjust tool actions. The method interacts with simulated users and learns from final completion rather than treating user input as static.
Illustrative example: A new constraint in turn two should change later tool arguments.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Core mechanism
Why taper expert supervision instead of switching off SFT at once?
Show answer and example
Answer: Early guidance stabilizes exploration, but prolonged imitation constrains it. Annealing and uncertainty weights allow a gradual transition while monitoring source shift. Blending isn't universally better; local proxy isn't an uncertainty model.
Illustrative example: Give learner less expert help as it improves rather than always equal help or an abrupt cutoff.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Implementation details
Does local CHORD gold candidate model real token uncertainty?
Show answer and example
Answer: No. Deterministic expert proxy; no LLM uncertainty model. Track expert off-policy vs student rollout and anneal as specified; keep expert answers out of evaluation inputs.
Illustrative example: Fixed candidate labels don't estimate uncertainty at every long-text position.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Method comparison
Is CHORD the same as PIVOT's per-question switching?
Show answer and example
Answer: No. CHORD blends losses with annealed supervision; PIVOT routes prompts by teacher perplexity to separate objectives. Granularity and schedule differ.
Illustrative example: One can apply both losses to a prompt; the other chooses its route.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Basics
Why anneal CHORD SFT auxiliary weight during RL?
Show answer and example
Answer: Expert demonstrations stabilize early exploration; excessive later imitation can restrict it. Dynamic weighting and token uncertainty ease the handoff without abrupt shift.
Illustrative example: Begin with more standard examples, then let outcome reward guide increasingly.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
Why can GPPO have PPO-like forward values and nonzero clipped-region gradients?
Show answer and example
Answer: Stop-gradient constructions decouple clipping values and derivatives. Boundary weights restore learning in flattened quadrants for positive exploration and negative examples. Inspect gradients, not just loss.
Illustrative example: Two operators can print one clipped score but only one has a boundary-weighted derivative.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
Is equality of clipped GPPO/PPO loss values enough to test implementation?
Show answer and example
Answer: No. Verify gradients in out-of-bound advantage quadrants against PPO's zero gradients, and explicit detach boundaries. No paper-scale long-CoT training.
Illustrative example: Positive advantage and a ratio above the upper bound directly test the recovered path.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
Why can identical forward losses produce different updates?
Show answer and example
Answer: Stop-gradient can change derivatives without changing values. GPPO separates forward and backward, so scalar logs cannot verify it.
Illustrative example: Two functions with equal output can have different derivatives through detach.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
Which PPO gradients does GPPO recover?
Show answer and example
Answer: Ordinary clipping can stop gradients for positive-advantage high ratios and negative-advantage low ratios. GPPO restores backward signal while keeping the forward value.
Illustrative example: A bad action already made rarer may still need an avoidance gradient.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Structured Decisions / Typed Readout: From Generated Sentences to Candidate Distributions
Core mechanism
Why can't runtime GLiClass reduce to a fixed text-only classifier?
GLiClass: Generalist Lightweight Model for Sequence Classification Tasks
Show answer and example
Answer: Scoring must use candidate content to handle changing label sets; a fixed index cannot convey new descriptions. Joint candidate scoring avoids per-candidate calls and text parsing. Local bilinear/centered scoring checks the mechanism without full weights or training.
Illustrative example: Example: The same inquiry may require different choices in account and credit-card taxonomies. New labels need descriptions, not unseen numbers.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Structured Decisions / Typed Readout: From Generated Sentences to Candidate Distributions
Implementation details
What does local rival-centered GLiClass center?
GLiClass: Generalist Lightweight Model for Sequence Classification Tasks
Show answer and example
Answer: Representations of candidates within the same question, before scoring their interaction with state. Other requests do not enter the mean. This tests relative candidate interaction, not original weights or PPO.
Illustrative example: Example: Refund and shipping candidates share their ticket's center, not labels from another ticket.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Structured Decisions / Typed Readout: From Generated Sentences to Candidate Distributions
Method comparison
How does GLiClass dynamic scoring differ from pairwise cross-encoders?
GLiClass: Generalist Lightweight Model for Sequence Classification Tasks
Show answer and example
Answer: Pairwise methods infer separately for each text-label pair. This interface processes candidates together and emits a joint distribution. Calls and normalization differ. The local lightweight interaction kernel is not a full Transformer or a speed benchmark.
Illustrative example: Example: Ten labels do not necessarily require ten independent LM calls; actual throughput depends on implementation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Structured Decisions / Typed Readout: From Generated Sentences to Candidate Distributions
Basics
Does a new GLiClass runtime label create a new classifier output neuron?
GLiClass: Generalist Lightweight Model for Sequence Classification Tasks
Show answer and example
Answer: No. Labels are input text scored for the current request. Candidate descriptions/set change, not a fixed output parameter per label. Semantics must be carried by content and still require validation; allowing new labels is not arbitrary domain expertise.
Illustrative example: Example: Replace the refund inquiry description with shipping follow-up, rather than reusing a class number with permanent meaning.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Core mechanism
Why use answer-conditioned trajectory construction only in data generation, not as evaluated agent input?
ToolGrad: Efficient Tool-Use Dataset Generation with Textual Gradients
Show answer and example
Answer: Known answers support backward construction of tool chains and textual-gradient repair to reduce training annotation. Revealing the target value in evaluation leaks the solution and prevents measuring independent evidence gathering. Training construction information must remain separate from deployed inputs.
Illustrative example: Example: Use known population to construct API traces for training. At test, give the population question without the reference number.
Sections in the Chinese source: 背景与主要改动, 核心公式或操作. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Implementation details
Do local answer-first and deduplicated edits cover ToolGrad-12B and BFCL?
ToolGrad: Efficient Tool-Use Dataset Generation with Textual Gradients
Show answer and example
Answer: No. Local references execute backward route construction and deduplicated text repair, without training 12B or running BFCL. Answers stay training-only, execution outcomes must be real, and failures are not backfilled. State transitions do not establish generated-data quality or downstream skill.
Illustrative example: Example: Deduplicating two corrections of the same bad argument verifies the edit rule, not automatic correct traces for unknown APIs.
Sections in the Chinese source: 核心公式或操作, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Method comparison
How does textual gradient differ from a numerical gradient when repairing failed calls?
ToolGrad: Efficient Tool-Use Dataset Generation with Textual Gradients
Show answer and example
Answer: Textual feedback identifies bad arguments, tools, or order and suggests edits; it is not differentiable backpropagation through discrete calls. It depends on feedback quality and execution verification. The name does not imply end-to-end parameter updates, and an edit is not a rerun.
Illustrative example: Example: Changing a date format from feedback is text editing. Reinvoke the tool and check its response to verify repair.
Sections in the Chinese source: 背景与主要改动, 核心公式或操作. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Basics
Why does ToolGrad target the cost of high-quality tool trajectories?
ToolGrad: Efficient Tool-Use Dataset Generation with Textual Gradients
Show answer and example
Answer: Multi-step calls need correct interfaces, arguments, and linked results. Human demos cost time and direct model generation can accumulate errors. ToolGrad uses target feedback during dataset construction to repair executable traces. Construction targets must not leak to evaluated agents.
Illustrative example: Example: During data creation, verify queries reach a known result and fix parameters. At test, gather evidence without reading the answer first.
Sections in the Chinese source: 背景与主要改动, 核心公式或操作. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
Why upper-clip engine ratios but keep small weights in TIS?
Show answer and example
Answer: Correction targets the training distribution. Huge weights can dominate variance; cap them. Small ratios indicate smaller target contributions, not automatic invalidity. This is not PPO policy ratio or two-sided masking.
Illustrative example: A token common in rollout but rare in training should contribute less without necessarily being removed.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
Which probability is the TIS denominator?
Show answer and example
Answer: Use the actual rollout engine probability below corresponding training probability; a separate policy ratio may follow. Local deterministic perturbations aren't vLLM/FSDP validation.
Illustrative example: Reversing the ratio can upweight an example that should be downweighted.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
Why does TIS upper-clip instead of two-sided dropping like IcePop?
Show answer and example
Answer: Capping large weights controls heavy tails while retaining small contributions. Two-sided masking drops severe mismatch in either direction. Zeroing small TIS weights changes the method.
Illustrative example: A tiny TIS ratio still contributes a little.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
Why use TIS for equal-weight training and inference engines?
Show answer and example
Answer: Precision and kernels can change probabilities, so rollout and training rescoring distributions differ. The ratio corrects implementation mismatch, not task rewards.
Illustrative example: Two compute paths using identical weights can assign slightly different token likelihoods.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
Why is separating training and execution more than storing logs on a different machine?
Show answer and example
Answer: A common transition contract converts framework behavior into attributable MDP data for return and advantage calculation, while execution keeps its tool orchestration. Missing state/action boundaries or mixed environment tokens prevent correct updates despite physical separation. Decoupling alone does not establish improvement for any agent.
Illustrative example: A trainer can update from standardized query actions and results. An unsegmented chat transcript is not directly an action sequence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
Which policy does the local Agent Lightning update using real test rewards?
Show answer and example
Answer: The code sandbox uses test exit codes to credit patches, updates a bounded patch policy, and reuses it by bug family. It does not run LightningRL on LLM parameters. Check command, edit, and test events separately from rewards and memory use.
Illustrative example: Failed tests can change later patch choices without updating LLM weights with RL.
Sections in the Chinese source: 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
How does execution/training decoupling differ from asynchronously collecting logs?
Show answer and example
Answer: Decoupling also maps traces into trainable transitions, assigns credit, and sends updated policies back to execution. Asynchronous logging only records and transmits. Without an execution path consuming updates, the loop is not connected.
Illustrative example: Growing logs while every subsequent run uses an unchanged policy does not let training affect execution.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why does Agent Lightning first convert execution events into common transitions?
Show answer and example
Answer: Agent runtimes log differently. Common MDP transitions and credit decomposition let training interpret states, actions, and rewards independently of execution, not just store logs elsewhere.
Illustrative example: A failed test after a code edit should connect to that patch action and state, not remain an isolated log message.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why freeze SID but learn behavioral residuals with zero-initialized MBA?
SaviorRec: Semantic-Behavior Alignment for Cold-Start Recommendation
Show answer and example
Answer: Behavioral content training and quantization form stable semantic IDs. Changing IDs would disrupt historical mapping. MBA gradually adapts behavior per code, initially without changing embeddings. Stable codes can limit response to large shifts.
Illustrative example: Keep camping IDs stable while preferences shift toward winter equipment.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
Why initialize MBA at zero, and what does bidirectional target attention read?
SaviorRec: Semantic-Behavior Alignment for Cold-Start Recommendation
Show answer and example
Answer: Zero residuals initially preserve frozen semantics, then learn per SID. Candidate queries history and conditions historical representations; this is not access to future outcomes.
Illustrative example: A candidate can guide matching without seeing the user's later click.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
How do behavior pairs differ from content-only similarity?
SaviorRec: Semantic-Behavior Alignment for Cold-Start Recommendation
Show answer and example
Answer: Interactions adapt the space to recommendation before quantization; content similarity may only reflect surface appearance. Behavior can be exposure-biased, and a later residual does not implement encoder training.
Illustrative example: Products often chosen together can align despite looking different.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why train SaviorRec's content encoder with behavior first?
SaviorRec: Semantic-Behavior Alignment for Cold-Start Recommendation
Show answer and example
Answer: Generic image/text similarity differs from co-click or co-purchase. SaviorEnc uses actual interactions to shape content before SID quantization. Cold start still depends on features and coverage.
Illustrative example: Visually similar products may meet different needs that behavior reveals.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Core mechanism
Why use an SAE encoder for topic probabilities but a separate decoder for steering?
Model Directions, Not Words: Mechanistic Topic Models Using Sparse Autoencoders
Show answer and example
Answer: The encoder maps topic directions to feature activation probabilities for a binomial count likelihood. The decoder maps feature combinations into residual space for intervention. Do not tie them implicitly. Steering removes the old parallel component before adding desired strength. Local work omits large-model experiments.
Illustrative example: Estimate sparse feature occurrence, then decode an intervention direction; an equally shaped encoder weight is not a substitute.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Implementation details
Does ETM steering strength zero leave activations unchanged?
Model Directions, Not Words: Mechanistic Topic Models Using Sparse Autoencoders
Show answer and example
Answer: Not necessarily. Remove the existing parallel component, add λs, and restore decoder bias. λ=0 still removes the component. Use an independent decoder, not encoder-tied weights.
Illustrative example: A strong original topic projection is removed at zero strength rather than preserved.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Method comparison
Why model feature counts with binomial rather than ordinary multinomial bags?
Model Directions, Not Words: Mechanistic Topic Models Using Sparse Autoencoders
Show answer and example
Answer: Each feature can occur once per token but features are not mutually exclusive. Model each count with N_tokens trials instead of forcing total counts to token count.
Illustrative example: One token activating two features can make total feature count exceed number of tokens.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Foundation models / Capability evidence: sampling, readout, and reliability
Basics
Why observe SAE features instead of words in Mechanistic ETM?
Model Directions, Not Words: Mechanistic Topic Models Using Sparse Autoencoders
Show answer and example
Answer: Sparse activations express model-internal concepts. A frozen SAE encoder maps topics to feature probabilities and fits document proportions to counts.
Illustrative example: Different words may activate similar concepts, and one word need not indicate the same topic in every context.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why protect workflow-required tools before considering recency or historic success?
Show answer and example
Answer: Retention scores reflect past usage and may undervalue an uncalled but required future tool. Protect dependencies first, then evict lower-scoring unprotected entries. If required tools exceed capacity, silent truncation does not preserve a valid workflow.
Illustrative example: Example: Upload is required at the end of archiving even if not recently used. Do not remove it solely for low recency.
Sections in the Chinese source: 背景与主要改动, 核心公式与算法. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
How should recency and historical success scores evict unprotected tools?
Show answer and example
Answer: Restrict candidates to those outside the protected workflow set. Keep scores λ times recency plus (1−λ) times success; evict the lowest. The weight trades off recency and history without redefining protection. Local deterministic descriptions and eviction counts do not establish live tool completion.
Illustrative example: Example: One evictable tool was just used but often fails; another is older but reliable. λ controls the choice, and required tools do not enter this ranking.
Sections in the Chinese source: 核心公式与算法, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
How do autonomous, workflow, and hybrid tool-description policies make decisions?
Show answer and example
Answer: Autonomous lets the agent decide retention; workflow uses current process requirements; hybrid protects those requirements and dynamically manages remaining tools. Freedom and structural constraints differ. Removal rate is insufficient if eviction prevents completion.
Illustrative example: Example: An agent may discard rare payments before one is needed. Workflow information protects payment while hybrid manages other entries.
Sections in the Chinese source: 背景与主要改动, 核心公式与算法. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Why do tool descriptions need bounded dynamic memory?
Show answer and example
Answer: MCP schemas and instructions consume context, while static truncation can remove tools in use. MemTool protects current workflow requirements and manages other descriptions dynamically. Preserve executable interfaces rather than maximizing removals.
Illustrative example: Example: An order lookup followed by refund still needs the refund tool even if it was not called recently.
Sections in the Chinese source: 背景与主要改动, 核心公式与算法. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
Why average token log ratios before exponentiation and clipping?
Show answer and example
Answer: One outcome belongs to one answer. One geometric weight gives all tokens the same relative advantage and clipping decision despite local noise. It changes credit aggregation and doesn't eliminate all length/MoE mismatches.
Illustrative example: A correct answer with one anomalous ratio gets different credit under token clipping and shared weighting.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
Should GSPO clip per token or per response?
Show answer and example
Answer: Clip one ratio per response, shared across tokens. Mask padding from length. Candidate experiments do not reproduce real token rollouts, MoE routes or clusters.
Illustrative example: Clipping each token and averaging usually differs from averaging then clipping.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
Is GSPO's geometric-mean ratio the actual sequence probability ratio?
Show answer and example
Answer: No. GSPO exponentiates the mean log ratio; the exact sequence ratio exponentiates the sum, multiplying token ratios. Averaging changes length scaling and the objective.
Illustrative example: Repeated small shifts accumulate in a product but the geometric mean measures typical shift.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
Why use one importance ratio for a full response in GSPO?
Show answer and example
Answer: Terminal reward is sequence-level, and token outliers can cause fragmented clipping. Average token log ratios to form one response weight and avoid local outlier dominance.
Illustrative example: One rare token's large shift need not determine distinct clips throughout a long reply.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
Why need proper scoring in addition to correctness rewards?
Beyond Binary Rewards: Training LMs to Reason About Their Uncertainty
Show answer and example
Answer: Binary reward cannot distinguish justified caution from false certainty. Proper scoring couples probabilities with outcomes to incentivize honesty. Report accuracy and calibration separately; local Brier gradients aren't full RL.
Illustrative example: A quiz bot claiming certainty on unfamiliar items can be penalized for that confidence even when ordinary rewards ignore it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
Which classes receive local RLCR Brier gradients?
Beyond Binary Rewards: Training LMs to Reason About Their Uncertainty
Show answer and example
Answer: Compute squared differences between the complete softmax distribution and one-hot target and backpropagate over all candidates, not just the displayed choice. Hybrid includes CE; no paper-checkpoint RL.
Illustrative example: Unselected classes consume probability mass and contribute to scoring.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
How does RLCR differ from displaying confidence after training?
Beyond Binary Rewards: Training LMs to Reason About Their Uncertainty
Show answer and example
Answer: A displayed field needn't affect optimization; RLCR incorporates a scoring rule. Local Brier/hybrid objectives are mechanism evidence, not full rollouts.
Illustrative example: A 90% badge doesn't mean bad calibration was penalized in training.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
Why predict both an answer and confidence in RLCR?
Beyond Binary Rewards: Training LMs to Reason About Their Uncertainty
Show answer and example
Answer: Correctness-only reward does not incentivize honest probabilities. A proper score evaluates probability and outcome, rewarding truthful probabilities in expectation under its assumptions.
Illustrative example: A model that often errs should report moderate confidence instead of certainty.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why require H=T and divisible hidden dimension?
RankMixer: Scaling Up Ranking Models in Industrial Recommenders
Show answer and example
Answer: Splitting T tokens of width D into H slices gives H outputs of width TD/H. H=T restores T×D for residual addition; D must divide by H.
Illustrative example: Four 64-wide tokens split into four 16-wide slices; three heads cannot split 64 evenly.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
How do independent FFNs learn interactions after fixed TokenMix?
RankMixer: Scaling Up Ranking Models in Industrial Recommenders
Show answer and example
Answer: Split every input by head and concatenate matching-index slices into new tokens, so specialized nonlinear FFNs receive cross-field signals. H=T and divisible width are layout conditions. Field mixing does not forbid sequence attention.
Illustrative example: User, product, and context slices can jointly enter an FFN without similarity weights.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Do RankMixer per-token FFNs attach to original fields or mixed positions?
RankMixer: Scaling Up Ranking Models in Industrial Recommenders
Show answer and example
Answer: Each FFN sees a mixed token containing same-index slices from all fields. Its parameters specialize in that subspace, not one original field. Preserve field/head/output layout.
Illustrative example: The first mixed FFN processes parts of all four fields, not only the first one.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
What do ReLU routing and DTSI solve beyond fixed top-k?
RankMixer: Scaling Up Ranking Models in Industrial Recommenders
Show answer and example
Answer: Sparse ReLU permits variable activated experts. Separate training and inference routers apply sparsity only on inference so experts can learn without starvation. Local sparse code does not validate full dual routing.
Illustrative example: Complex tokens can use more experts while rarely deployed experts still train.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
Why not credit TokenMix alone for the dense versus shared-FFN gain?
RankMixer: Scaling Up Ranking Models in Industrial Recommenders
Show answer and example
Answer: Both arms use TokenMix; dedicated versus shared FFNs and different parameter counts change together. The sparse arm also trails dense. Report budgets and popularity bias; local results do not establish paper MFU or A/B.
Illustrative example: Both cars already use the same transmission, so a larger engine's gain does not isolate it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Which attention does RankMixer replace?
RankMixer: Scaling Up Ranking Models in Industrial Recommenders
Show answer and example
Answer: It replaces attention for heterogeneous field interactions, not necessarily independent behavior sequence encoding. TokenMix and specialized FFNs process fields after feature extraction.
Illustrative example: History attention can extract photography interest before mixing it with price and item fields.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
How does feedback-then-item differ from direct next-item prediction?
Scaling Recommender Transformers to One Billion Parameters
Show answer and example
Answer: ARGUS factors joint action into feedback type then conditional item, sharing a trunk while preserving feedback distinctions. Inference cannot read future feedback. Local module ablation is negative.
Illustrative example: Expected long watch and quick skip can favor different items for the same history.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why distinguish DIN and next-item-only baselines?
Scaling Recommender Transformers to One Billion Parameters
Show answer and example
Answer: DIN compares whole models; matched next-item-only isolates feedback factorization. Both local comparisons lack gains, and neither should hide the other's negative evidence.
Illustrative example: Overall proximity to DIN can coexist with degradation after adding the feedback path.
Sections in the Chinese source: 本地复现. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does conditional factorization differ from a parallel auxiliary head?
Scaling Recommender Transformers to One Billion Parameters
Show answer and example
Answer: A parallel head adds loss without necessarily changing item prediction. ARGUS explicitly conditions items on feedback, so feedback errors can propagate. Measure both.
Illustrative example: Separate “will like” and item outputs are multitask, not a feedback-conditioned generator.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
How read ARGUS's feedback–item joint distribution?
Scaling Recommender Transformers to One Billion Parameters
Show answer and example
Answer: Predict feedback from history, then condition item probability on it and multiply. True feedback is training supervision, not an inference input unless specified externally under protocol.
Illustrative example: Likes can favor relatable clips while long watches favor tutorials.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Core mechanism
Why not attribute a purchase of B entirely to clicked item A as a conversion of A itself?
Click A, Buy B: Rethinking Conversion Attribution in E-Commerce Recommendations
Show answer and example
Answer: The clicked and purchased objects can differ; same-item last-click attribution misses cross-item paths. Separate Click-A-Buy-A and Click-A-Buy-B tasks and use taxonomy to describe relations between A and B. This improves observed attribution and prediction, not causal identification.
Illustrative example: A user views an entry-level camera and buys an upgrade. Same-item counting marks the first camera as worthless despite its role in the decision path.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Implementation details
How should a CABB viewing-sequence proxy be reported without inventing purchase labels?
Click A, Buy B: Rethinking Conversion Attribution in E-Commerce Recommendations
Show answer and example
Answer: State that sequential views proxy Click-A/Buy-B, compare matched data, candidates, and baselines, and limit conclusions to the proxy. Taxonomy weights connect corresponding items and must not use test targets to decide candidate categories. Public ratings are not transaction amounts.
Illustrative example: Report simulated cross-item paths from successive views, not that users really clicked A and bought B. A target category known only afterward is not an inference input.
Sections in the Chinese source: 背景与主要改动, 本地复现. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Method comparison
Why does CABB need behavior types and item hierarchy beyond ordinary next-item prediction?
Click A, Buy B: Rethinking Conversion Attribution in E-Commerce Recommendations
Show answer and example
Answer: Adjacency alone need not distinguish browsing, clicking, and purchase or describe the A–B relationship. CABB explicitly supervises cross-item purchase paths and uses taxonomy to organize transferable relations. Sequential viewing preserves adjacency only and does not validate real purchase attribution.
Illustrative example: Watching movie B after A can test cross-item connections, but there is no ad-click and transaction loop.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Basics
Why does Click-A-Buy-B distinguish clicked and purchased items?
Click A, Buy B: Rethinking Conversion Attribution in E-Commerce Recommendations
Show answer and example
Answer: An item can reveal a need and another item satisfy it through price or specs. Separate CABA and CABB supervision retains direct conversions and cross-item guidance; taxonomy offers relation cues. Observed co-occurrence does not prove that A causes purchase of B.
Illustrative example: Clicking one camera brand and buying another is not a conversion of A, but A can still participate in the decision.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
How does PinFM DCAT retain early candidate fusion without re-encoding user context per item?
PinFM: Foundation Model for User Activity Sequences at a Billion-scale Visual Discovery Platform
Show answer and example
Answer: Early fusion lets candidates interact with history internally. DCAT self-attends once over deduplicated context, caches layer K/V, and lets candidate queries read it. Shared state must be candidate-independent. Serving savings and pretraining gains are distinct; retain local negative outcomes.
Illustrative example: Illustration: Encode 500 events once for 100 candidates rather than copying all history 100 times, while candidates still read it inside layers.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why confirm actual context K/V reuse during DCAT full-catalog inference?
PinFM: Foundation Model for User Activity Sequences at a Billion-scale Visual Discovery Platform
Show answer and example
Answer: Compute context self-attention once and let candidate layers perform only legal candidate-dependent work. Equal scores do not imply equal execution cost, so check call counts and dependencies. Bind caches to history and version and exclude candidate-conditioned states.
Illustrative example: Illustration: Recomputing history for 1,000 candidates can give the same NDCG while yielding no amortization.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
Why does scratch versus pretrained DCAT answer a different question from DCAT versus ordinary Transformer?
PinFM: Foundation Model for User Activity Sequences at a Billion-scale Visual Discovery Platform
Show answer and example
Answer: Keeping DCAT fixed isolates pretraining, while changing architecture tests cache decomposition or fusion. The selected local pretrained model regressed on test; mathematical DCAT fidelity does not establish pretraining value. Measure serving efficiency separately.
Illustrative example: Illustration: Training two models with the same architecture differently says nothing about a third architecture that was never tested.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why combine next-, multi-, and future-token objectives in PinFM?
PinFM: Foundation Model for User Activity Sequences at a Billion-scale Visual Discovery Platform
Show answer and example
Answer: The immediate item, future window, and future from a specified position supply distinct time scales, helping learn more than recency before transfer. Future labels are training-only. Cross-scenario pretraining can negatively transfer despite being a foundation model.
Illustrative example: Illustration: Next click teaches immediate intent, a window teaches coexisting interests, and later targets add long-term direction.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Core mechanism
Why require automatically decidable game rules in zero-sum SPIRAL?
Show answer and example
Answer: Adversarial progress creates curriculum, but independent rules must decide wins. Zero-sum incentives do not guarantee broad reasoning coverage. Local state/objective checks don't reproduce self-play training.
Illustrative example: The proposer can't change rules midgame to make every solution lose.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Implementation details
Why preserve zero-sum rewards across SPIRAL roles?
Show answer and example
Answer: One outcome gives opposite credit under aligned states/rules. Rewarding both for the same victory changes the game. Local kernel isn't large multi-round training.
Illustrative example: A proposer win cannot also be a solver win.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Method comparison
What are limits of self-play vs fixed hard-problem banks?
Show answer and example
Answer: Self-play adapts but can get trapped in game-specific tricks. Fixed banks have defined coverage but no evolving opponent. Game gains don't imply general reasoning.
Illustrative example: Winning a word game needn't improve math, retrieval or tools.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Basics
Why let one SPIRAL model propose and solve games?
Show answer and example
Answer: An adversarial, decidable zero-sum game creates a curriculum as opponents evolve. Roles need distinct rewards and explicit rules; proposer cannot declare outcomes.
Illustrative example: One role creates a position, another tries to solve it, and a program judges.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
Why does CISPO retain gradients for some tokens PPO would clip flat?
Show answer and example
Answer: Clipped weights scale log-probability gradients while retaining advantage direction. PPO's minimum is flat in certain quadrants. Test gradients, not only values or clipped advantages.
Illustrative example: A rare negative-advantage token can still receive bounded suppression even if PPO is flat.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
What if CISPO differentiates directly through the clipped ratio?
Show answer and example
Answer: Extra weight derivatives change the policy-gradient objective. Keep rollout, weight construction and logp path separate. Local checks do not reproduce MiniMax training.
Illustrative example: Current action logp should supply gradient without duplicating it through the weight branch.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
Why aren't CISPO and PPO the same because both clip?
Show answer and example
Answer: PPO takes the minimum of clipped and unclipped surrogates. CISPO multiplies logp and advantage by clipped weight. Backward and out-of-range behavior differ despite matching bounds.
Illustrative example: One boundary can stop gradients; the other preserves updates.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
Why clip importance weights in CISPO?
Show answer and example
Answer: Bound magnitude while preserving advantage direction and learning, unlike PPO min-surrogate flat regions. Separate frozen weights from differentiable logp.
Illustrative example: A capped positive weight can still reinforce through logp.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Core mechanism
What does MGOE's explicit task graph add to MMoE's sample-level gate?
Macro Graph of Experts for Billion-Scale Multi-Task Recommendation
Show answer and example
Answer: Sample-level mixtures do not explicitly encode stable task relationships. Graph bottom and experts propagate between related objectives before task towers, separating global structure from local routing. Wrong edges can propagate negative transfer. Compare related and unrelated tasks rather than assuming the graph is correct.
Illustrative example: Illustration: Clicks, cart additions, and purchases may share paths, while reports need other signals; the gate need not relearn every task relationship per sample.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Implementation details
Why align MGOE's normalized adjacency and task indices with expert tensors?
Macro Graph of Experts for Billion-Scale Multi-Task Recommendation
Show answer and example
Answer: Rows and columns of A identify nodes and task towers must read the correct propagated rows of H. Normalization, self-loops, and depth determine mixing. Use labels only in training; do not rebuild the graph from test feedback.
Illustrative example: Illustration: A purchase node paired with watch-time features still multiplies correctly but shares the wrong information.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Method comparison
What additional sharing bias does MGOE introduce relative to independent MMoE task gates?
Macro Graph of Experts for Billion-Scale Multi-Task Recommendation
Show answer and example
Answer: MMoE shares experts implicitly by example. MGOE adds graph-constrained task propagation, which can help sparse objectives or transmit wrong relations. Overall gains and changed head share do not establish all-task improvement.
Illustrative example: Illustration: A sparse purchase task can borrow click knowledge; a mistakenly connected returns task may inherit the wrong direction.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Basics
Why build a Macro Task Merging Graph before expert propagation in MGOE?
Macro Graph of Experts for Billion-Scale Multi-Task Recommendation
Show answer and example
Answer: The graph represents stable sharing relations that a per-sample gate does not explicitly encode. Macro Graph Bottom/Experts propagate information before task-specific towers. Relations can change across scenarios, and edges do not establish causality.
Illustrative example: Illustration: Click, watch time, and purchase can share information without implying clicks cause purchases.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Core mechanism
Why use asynchronous post-request ranking to supplement retrieval for the next RADAR request?
RADAR: Recall Augmentation through Deferred Asynchronous Retrieval
Show answer and example
Answer: A strong ranker cannot usually scan the entire catalog within request latency. RADAR does the expensive scan afterward, caches good candidates, and merges them with live retrieval next time. Old history makes caches stale, so live retrieval still captures new intent.
Illustrative example: Illustration: Precompute likely movies after one visit and add them on the next, while live retrieval uses newly observed clicks.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Implementation details
Why deduplicate and recheck eligibility when merging RADAR live and cached top-k?
RADAR: Recall Augmentation through Deferred Asynchronous Retrieval
Show answer and example
Answer: Old items may be unavailable, consumed, or ineligible. Merge by item ID and preserve provenance for incremental evaluation. Record cache generation time and history cutoff; the next target cannot enter the async job ahead of time.
Illustrative example: Illustration: Drop a listing that filled since yesterday and count a movie retrieved by both channels only once.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Method comparison
How does RADAR's prior-request cache differ from a predicted future pair-latent cache?
RADAR: Recall Augmentation through Deferred Asynchronous Retrieval
Show answer and example
Answer: RADAR caches selected item IDs under previous history. Pair-latent caching stores intermediate interactions for specific user–item pairs. Reuse, misses, and invalidation differ, so one hit rate cannot represent both costs or establish a quality ordering.
Illustrative example: Illustration: Reusing a movie list saves nomination; intermediate vectors are different artifacts and may still need current scoring.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Automated research and efficient serving
Basics
Why is RADAR's cached top-k a nomination rather than guaranteed exposure?
RADAR: Recall Augmentation through Deferred Asynchronous Retrieval
Show answer and example
Answer: It was selected under previous history and still competes with live candidates under current needs and eligibility. A cached score is neither exposure nor a click. Distinguish nominations to assess freshness and downstream value.
Illustrative example: Illustration: A cached movie can be filtered because the user watched it or now wants children's content.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why keep raw recent events and add Next Action Loss?
TransAct V2: Lifelong User Action Sequence Modeling on Pinterest Recommendation
Show answer and example
Answer: Candidate matching retrieves old evidence but can miss emerging intent. Keep recent events and fuse with retrieved history. NAL learns next positive action beyond CTR; missed remote events and serving performance remain separate.
Illustrative example: A tent candidate finds old camping while raw history shows new family-travel interest.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Can Next Action Loss labels enter the current input?
TransAct V2: Lifelong User Action Sequence Modeling on Pinterest Recommendation
Show answer and example
Answer: No. Next positive behavior is supervision only; input reads the past, and negatives must be valid. Local model scope excludes production SKUT/request dedup and throughput.
Illustrative example: Adding the future click to history trivially leaks the auxiliary target.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does guaranteed recent retention differ from full-history top-k?
TransAct V2: Lifelong User Action Sequence Modeling on Pinterest Recommendation
Show answer and example
Answer: It keeps fresh behavior even with low similarity, while other positions come from candidate search. Extra context costs compute; gains depend on data.
Illustrative example: A new interest may not rank highly under an old similarity model.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why combine lifelong and recent histories in TransAct V2?
TransAct V2: Lifelong User Action Sequence Modeling on Pinterest Recommendation
Show answer and example
Answer: Candidate retrieval finds relevant older evidence, while recent history captures immediate intent. One Transformer fuses both with candidates early.
Illustrative example: A current browse and a similar older interest can both support a candidate.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Core mechanism
Why retain negative Shapley values and credit only the span's final token?
SCAR: Shapley Credit Assignment for More Efficient RLHF
Show answer and example
Answer: A span may reduce frozen RM value, so clipping breaks decomposition. End-token placement and original-order coalitions define reward scattering. Total equivalence has discount conditions; small exact enumeration isn't Owen approximation or full RLHF.
Illustrative example: A contradictory extra paragraph can reduce value; zeroing its negative credit breaks total equality.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Implementation details
Why place SCAR credit at span end and preserve masked positions?
SCAR: Shapley Credit Assignment for More Efficient RLHF
Show answer and example
Answer: Score coalitions in original order, filling excluded positions with spaces, and assign credit to each span's last token. Keep negatives and zero empty value. Reward conservation requires the corresponding γ=1 condition.
Illustrative example: Joining text across removed middle spans changes the defined coalition.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Method comparison
How does the exact local Shapley component differ from the paper's SHAP/Owen pipeline?
SCAR: Shapley Credit Assignment for More Efficient RLHF
Show answer and example
Answer: Enumeration is exponential and accepts small complete value tables only. The full method also has adaptive segmentation and hierarchical approximation.
Illustrative example: Three spans permit all coalitions; dozens of spans do not fit the same budget.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Basics
Why consider coalitional marginal contributions of spans in SCAR?
SCAR: Shapley Credit Assignment for More Efficient RLHF
Show answer and example
Answer: A span's value depends on other spans. Shapley weights gains over coalitions to allocate terminal reward, including negative contributions.
Illustrative example: An explanation can help only when its premise is also present.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Core mechanism
Can negative entropy replace INTUITOR self-certainty?
Show answer and example
Answer: No. The paper uses KL(U||p), unlike KL(p||U) or negative entropy; low-probability tails affect directions differently. Local sampled-candidate ratios aren't tokenwise reproduction. Confidence isn't correctness.
Illustrative example: Two distributions with similar maxima can have very different small tails, which KL directions weight differently.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Implementation details
Does current intuitor code implement full-vocabulary self-certainty?
Show answer and example
Answer: No. Reward uses sampled-candidate logp+log N and diagnostic KL(p||U), lacking token-level full-vocabulary KL(U||p). Historical scores remain diagnostics even if docs are corrected.
Illustrative example: Changing the documented direction does not alter the executable branch.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Method comparison
Can INTUITOR's intrinsic KL and reference KL be merged?
Show answer and example
Answer: No. Uniform-to-current KL measures certainty; reference KL anchors drift. They use different references and roles. Treating intrinsic reward as regularization changes the target.
Illustrative example: One encourages certainty while another discourages moving from old policy, potentially opposing each other.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Basics
Where does INTUITOR reward come from without an external verifier?
Show answer and example
Answer: Compute self-certainty from each prefix's full next-token distribution and average across the response. This is intrinsic signal, not correctness; use independent evaluation.
Illustrative example: A sharp distribution can still produce wrong arithmetic.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
Why does multi-turn web GRPO need dynamic context rather than repeated single-turn answer training?
Show answer and example
Answer: Clicks, inputs, and navigation change page state. Later actions need the current page and relevant history. Full rollouts connect terminal success to generated actions, and dynamic retention limits HTML growth. Behavior cloning and initialization also shape exploration; a group-advantage formula is not complete web interaction training.
Illustrative example: After login, a changed page requires reading the new form. Repeated answers from the first screenshot cannot teach that transition.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
How can compressed web state avoid treating old DOM as the current actionable page?
Show answer and example
Answer: Bind each action to the then-current observation and compressed context; a fixed initial page cannot substitute for dynamic state. The mini-suite uses deterministic text observations with no real DOM effects or Qwen/Llama updates.
Illustrative example: A button ID may become invalid after navigation even if it appears in history.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
What training problems do behavior-cloning warm-up and subsequent M-GRPO address?
Show answer and example
Answer: Warm-up establishes basic interaction and initialization. M-GRPO uses relative full-task success to optimize the policy. The stages have distinct supervision; a GRPO registry name cannot replace the paper's behavior and long-CoT initialization.
Illustrative example: First learn legal clicks and inputs, then improve long-horizon choices from task outcomes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why does WebAgent-R1 train on complete multi-turn web trajectories?
Show answer and example
Answer: State, HTML, and action history change during interaction, and success depends on multiple decisions. The method manages recent and relevant context dynamically, collects full traces, and updates from group-relative outcomes instead of optimizing isolated answers repeatedly.
Illustrative example: After clicking to a new page, choose the next action from the new state, not the initial page.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
Why compare micro advantages among actions from the same recurrent anchor state?
Show answer and example
Answer: Different states vary in available actions and remaining difficulty, confounding direct return comparisons. A shared anchor controls local conditions better. Micro credit complements trajectory-level macro advantage; stochastic continuation means differences are not noiseless causal effects or a replacement for terminal success.
Illustrative example: Compare looking for a key and pushing a locked door from the same point, not against a trajectory already at the goal.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
Why record anchor identity and the corresponding action with GiGPO micro advantage?
Show answer and example
Answer: The mean comes from the same G(s_t), so wrong keys or mixed states change the baseline. Keep macro/micro logs, actual group membership, and state-proxy boundaries, not only combined advantage.
Illustrative example: Two screens with the same title can have different task state and should not be merged by title alone.
Sections in the Chinese source: 核心公式, 复现边界. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
How do same-anchor micro groups differ from ranking all steps together?
Show answer and example
Answer: A shared state makes comparisons more comparable; mixing states confounds difficulty and consequences. Shared plan steps can be local proxies but do not establish identical underlying environment states.
Illustrative example: A high return in an easy state should not be the same baseline for a difficult one.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why does GiGPO need both macro trajectory advantage and micro step advantage?
Show answer and example
Answer: Terminal outcomes summarize the trajectory but do not identify useful steps. GiGPO compares trajectories, then actions grouped by recurrent anchor state, and adds global and local credit.
Illustrative example: Two trajectories can ultimately fail while one action from their shared intermediate state is more helpful.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why estimate prefix value with ordered regression instead of summing item CTRs?
A Generative Re-ranking Model for List-level Multi-objective Optimization at Taobao
Show answer and example
Answer: Item scores miss list interactions. Predict cumulative behavior thresholds after adding a candidate. Target queues narrow search, batched masks evaluate, and MMR participates during selection. Proxy labels and search bounds do not establish global optimality.
Illustrative example: A third similar phone may add less engagement than headphones despite higher standalone CTR.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Does local batched execution reproduce the paper's 19ms kernel?
A Generative Re-ranking Model for List-level Multi-objective Optimization at Taobao
Show answer and example
Answer: No. It is PyTorch masks without production optimization. MovieLens proxies clicks, trades, and upstream scores, not GMV. Align prefix targets and candidate masks.
Illustrative example: One batch call need not match a specialized kernel's device cost.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
How does MMR during generation differ from deduplication afterward?
A Generative Re-ranking Model for List-level Multi-objective Optimization at Taobao
Show answer and example
Answer: Choosing with marginal utility and similarity changes later prefixes. Removing duplicates only from the final list can leave holes or miss another generation trajectory.
Illustrative example: Deleting an early duplicate afterward does not recreate the alternative selection path.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Does ordered regression predict item IDs or cumulative behavior thresholds?
A Generative Re-ranking Model for List-level Multi-objective Optimization at Taobao
Show answer and example
Answer: It predicts whether a prefix reaches each click/payment count. Threshold and prefix-length axes are distinct; score the full prefix after adding a candidate.
Illustrative example: For three items, outputs can mean at least one or two clicks, not item identities.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Core mechanism
Why put the Gated Attention gate after SDPA rather than in logits?
Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free
Show answer and example
Answer: Aggregate each head first, then use an input-conditioned sigmoid before output projection. It controls how much of what was read is passed onward and adds nonlinearity between aggregation and projection. This is not changing historical softmax probabilities.
Illustrative example: Suppress a head's formatting output while retaining another head's answer evidence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Implementation details
Can a fixed constant replace a query-dependent per-head gate?
Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free
Show answer and example
Answer: A constant loses input dependence and head selectivity. Produce sigmoid gates from current hidden state and multiply SDPA output before projection. Local gate statistics only verify a nontrivial path.
Illustrative example: Different questions over the same article can use different gates instead of always multiplying by 0.5.
Sections in the Chinese source: 核心公式, 本地复现. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Method comparison
Why isn't a pre-SDPA sigmoid equivalent to a post-SDPA gate?
Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free
Show answer and example
Answer: Gating scores or values changes aggregation itself. The post-SDPA gate modulates the already aggregated output; placement is a documented ablation result.
Illustrative example: Scaling one contribution before combining differs from scaling the whole combined result.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Basics
Does the Gated Attention gate decide where to read or how much to emit?
Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free
Show answer and example
Answer: Attention first selects and aggregates positions. The current input then controls how much of each SDPA head output goes forward.
Illustrative example: A head can read a formatting marker but emit less when irrelevant.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why can larger LONGER merge factor K fail to reduce total FLOPs?
LONGER: Scaling Up Long Sequence Modeling in Industrial Recommenders
Show answer and example
Answer: Section 3.4 shortens positions while widening merged features, increasing projection cost but reducing attention cost. InnerTrans supplies within-group interaction. This is not fixed-width mean pooling or monotone quality.
Illustrative example: Moving from four to eight events per wider token can make matrix work rebound.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Do later LONGER layers keep updating long history after using it as KV?
LONGER: Scaling Up Long Sequence Modeling in Industrial Recommenders
Show answer and example
Answer: The first layer uses short global/recent Q against merged long KV. Output follows Q length; later self-attention runs only on the short states. This is inter-layer compression, not upfront truncation. The local namesake lacks full fidelity.
Illustrative example: Short query cards inspect a long reference once, then only the cards are refined.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why verify first-layer Q length and subsequent state length?
LONGER: Scaling Up Long Sequence Modeling in Industrial Recommenders
Show answer and example
Answer: The defining path is short global/recent Q over long KV, then short-only self-attention. Concatenating every input and self-attending each layer changes information flow. Inspect actual shapes and masks.
Illustrative example: With KV length 500 and Q length 100, the next layer should receive 100 positions, not over 600.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
Where does LONGER compress compared with candidate top-k history?
LONGER: Scaling Up Long Sequence Modeling in Industrial Recommenders
Show answer and example
Answer: Top-k discards events before fine encoding. LONGER lets short queries read merged long KV once, then propagates short output. Merging is still lossy and the local first-layer path is not fully faithful.
Illustrative example: Read a broad compressed source once rather than discard all but ten pages up front.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
How cache user KV if some global tokens contain candidates?
LONGER: Scaling Up Long Sequence Modeling in Industrial Recommenders
Show answer and example
Answer: Sections 3.5.2/3.6.3 mask candidate-specific globals from user sequences. Cache independent user KV and compute candidate query outputs separately. Global means aggregation role, not independence.
Illustrative example: Two products share behavior KV but use distinct globals; do not cache a user state already conditioned on product A for B.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why isn't a LONGER global token a fixed user summary?
LONGER: Scaling Up Long Sequence Modeling in Industrial Recommenders
Show answer and example
Answer: Globals can include CLS, user, and candidate fields. Short queries propagate their first-layer states; candidate globals remain request-specific. Cacheability depends on actual dependency.
Illustrative example: A user ID can be shared but a price-derived global belongs to one item.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why does merging actions into item tokens change HSTU-style ranking cost in GenRank?
Towards Large-scale Generative Ranking
Show answer and example
Answer: Interleaving doubles positions per event, increasing quadratic interactions. GenRank keeps action features or supervision on item positions and adds position/time biases. Fewer tokens do not guarantee equivalent expression; local quality changed alongside speed.
Illustrative example: Ten events become ten instead of twenty positions, reducing the attention matrix while potentially losing action dependencies.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why not input GenRank's current prediction target like historical actions?
Towards Large-scale Generative Ranking
Show answer and example
Answer: History actions are known and can accompany items; the current action is the label. Separate fields and causal masks and use only request-time information in position/time bias. Matching shapes do not rule out leakage.
Illustrative example: Include a prior camera click, not the true future click on the shoe being scored.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
Why compare latency and AUC for GenRank versus interleaved item/action ranking?
Towards Large-scale Generative Ranking
Show answer and example
Answer: Fewer positions reduce attention work but can sacrifice interaction expressivity. Local latency and AUC both decreased, so efficiency is not unchanged accuracy or superiority to HSTU. Hardware and candidates limit extrapolation.
Illustrative example: One hundred events can use 100 rather than 200 positions but lose action-order detail; speed alone hides that.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why can GenRank retain multi-action learning with items and actions at one position?
Towards Large-scale Generative Ranking
Show answer and example
Answer: Actions remain input features or supervision, just not separate positions. Position and time biases preserve temporal cues. This compresses organization, not behavioral relevance, and equivalence needs testing.
Illustrative example: An impression and its known click can share a position while clicks and skips remain different labels.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Core mechanism
Why does a self-generated Absolute Zero curriculum still need verification?
Show answer and example
Answer: Proposer creates tasks, solver answers and program execution supplies independent rewards, replacing human annotations. Verifiable tasks still supervise; exposing saved answers breaks the loop. Local diagnostics aren't full training.
Illustrative example: Generate executable code and predict output, then run it to grade instead of showing stored stdout.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Implementation details
Can Absolute Zero show solver answers saved during task generation?
Show answer and example
Answer: No. Keep execution results on verifier side; solver sees public task only. Local curriculum/state checks aren't large-model capability evidence.
Illustrative example: Don't append actual output to the program-output prediction prompt.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Method comparison
How does making harder tasks differ from a capability-boundary curriculum?
Show answer and example
Answer: Too hard can yield all failures and no signal. Use verifiable challenges near current ability. Legal format and novelty don't establish teaching value.
Illustrative example: A task no one can solve at the budget may be worse for learning than a moderate challenge.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Basics
Why does Absolute Zero still need an executable verifier?
Show answer and example
Answer: Generated tasks and solutions aren't truth. Program execution provides independent reward. No human item labels isn't no supervision; task executability and verifier quality bound scope.
Illustrative example: Run generated code rather than trust its claimed output.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
Why doesn't collapsed reward variance necessarily mean stable task learning?
Show answer and example
Answer: The policy may repeat failures or rigid behavior. Collapsed variance with gradient spikes signals Echo Trap risk. Trajectory filtering, a critic baseline, and decoupled clipping address data, estimates, and update stability separately. Real task performance is still required; low variance alone is not successful convergence.
Illustrative example: If all rollouts repeat one ineffective command, returns are zero with no variance. This is exploration collapse, not mature capability.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
Does a periodic local Echo Trap probe measure the paper's training-collapse curve?
Show answer and example
Answer: No. It makes a stability path observable. The policy selects among deterministic candidates and never updates LLM parameters. Distinguish probe triggers, actual gradients, and large-scale training curves; counts are not evidence of model collapse.
Illustrative example: Checking every few rounds does not imply that the paper's Echo Trap occurred in those rounds.
Sections in the Chinese source: 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
How does StarPO-S trajectory filtering differ from clipping ratios only inside PPO loss?
Show answer and example
Answer: Filtering controls which trajectories enter the objective. Ratio clipping limits drift for admitted samples, while the critic and gradient controls are separate. One clipping operation does not replace the complete stabilization chain.
Illustrative example: Repeated low-information trajectories can still dominate training samples even if their ratios are clipped.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why does RAGEN monitor Echo Trap separately?
Show answer and example
Answer: Multi-turn stochastic training can enter self-reinforcing repetition, with collapsed reward variance and gradient spikes. StarPO-S uses trajectory filtering, a critic baseline, and gradient/clipping stabilization rather than interpreting low variance as learning.
Illustrative example: A policy can consistently follow a failed path without any improvement.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
Why label TTRL as adaptation rather than fixed-model evaluation?
Show answer and example
Answer: Test inputs trigger sampling and immediate parameter updates, so the evaluated model changes. Report sampling/adaptation budgets and keep gold in independent scoring. Pseudo-label consensus can reinforce errors; unlabeled isn't no learning.
Illustrative example: Sampling and updating before answering differs from one call to the original model even without gold.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
May TTRL answer normalization see test gold?
Show answer and example
Answer: No. Equivalence and votes must use permitted output rules; gold is scored afterward. Local loops aren't paper-scale real rollouts.
Illustrative example: Do not filter by the correct number before claiming an autonomous vote.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
How does TTRL differ from majority-vote inference?
Show answer and example
Answer: Voting just aggregates outputs; TTRL updates parameters using pseudo-rewards and affects later behavior. Account for training compute and persistent state.
Illustrative example: After the same samples, voting returns; TTRL additionally optimizes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
Where does TTRL get rewards on unlabeled test tasks?
Show answer and example
Answer: Sample several answers, use majority consensus as a pseudo-label, reward agreement and update online. No gold, but consensus can be wrong; evaluate separately.
Illustrative example: Repeatedly guessing the same wrong number creates agreement, not truth.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
Why track behavior μ in mixed demonstration/rollout batches?
Show answer and example
Answer: Shared batch doesn't make sources on-policy. Importance ratios and regularization address source shift while keeping exploration. Students must still reach uncovered states; do not leak demos into evaluated inputs.
Illustrative example: Record expert success separately from failed student rollouts, not as a student sample.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
What roles do μ and ref have in LUFFY?
Show answer and example
Answer: μ describes mixture sampling; ref anchors KL. Track online vs demonstration provenance. Local candidates aren't paper text/distributed training.
Illustrative example: Expert data and initial-student reference need not have the same probabilities.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
How does LUFFY differ from inserting demonstrations into an ordinary GRPO group?
Show answer and example
Answer: External demonstrations have another sampling source and need correction/regularization. Treating them as student samples gives wrong denominator. Source-aware updates are more than adding good text.
Illustrative example: An inserted expert solution is not a lucky current-policy sample.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
Why combine offline demonstrations and online exploration in LUFFY?
Show answer and example
Answer: Demonstrations reveal paths students rarely sample; online rollouts preserve current exploration. Handle shared support and source probabilities rather than pure imitation or narrow exploration.
Illustrative example: Experts show a proof while the student still generates attempts and receives feedback.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Core mechanism
Why dynamically scale rewards by tool when their reward difficulties differ?
Show answer and example
Answer: High raw rewards from easy tools can dominate updates, burying improvements on harder tools. Center and scale by tool distribution to compare relative performance while retaining selection, argument, execution, and answer signals. Normalization does not fix wrong rewards or equalize actual difficulty.
Illustrative example: Example: Calculator successes are common and database queries often fail. Normalize separately so easy calculator rewards do not monopolize learning.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Implementation details
Do local reward and state updates prove full multi-tool RL training?
Show answer and example
Answer: No. Local tests cover paper objectives, transitions, and budgets without the original large model, multi-GPU RL, or full real-tool benchmark. Correct reward arithmetic still needs policy sampling, real external execution, and parameter updates. Registration and perfect mini-suites cannot fill those gaps.
Illustrative example: Example: Computing advantage from four reward dimensions tests an objective, not learned argument generation.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Method comparison
Why are tool-selection, execution, and answer rewards not substitutes?
Show answer and example
Answer: Selection checks task fit, execution checks whether a call runs, and answer reward checks final goal. Correct tools can fail from bad arguments, and successful calls can return irrelevant data. Separate signals expose different errors; any one high score does not establish end-to-end ability.
Illustrative example: Example: A database query runs with the wrong month. Selection and execution pass without deserving answer reward.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Basics
Why doesn't checking only ToolRL's final answer establish a good tool policy?
Show answer and example
Answer: The result conflates choosing tools, generating arguments, executing, and using outputs. Granular feedback constrains each behavior. Calibrate rewards without letting format or execution hide wrong answers. Local reward computation is not large-model RL.
Illustrative example: Example: A weather tool can be correct but use the wrong city. A single answer score cannot identify argument versus selection failure.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Core mechanism
Why not learn to avoid tools entirely when calls carry penalties?
Show answer and example
Answer: The objective also rewards correct answers. Fewer calls matter only if quality does not suffer. Alternating reasoning and actual feedback teaches when to outsource, recover, and stop. Optimizing call count alone may replace needed computation with fluent guesses.
Illustrative example: Example: Mental arithmetic can avoid calls, but complex statistics need code. One reliable result can be worth more than saving a call and answering incorrectly.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Implementation details
Can the model fill in a tool observation to complete a reasoning/action trajectory?
Show answer and example
Answer: No. Actual tool or environment execution must supply feedback. The model generates its reasoning and actions only. Fabricated outputs remove learning evidence for calls and corrections. Local controlled state/budget tests do not train the paper model or run real math benchmarks.
Illustrative example: Example: Writing 'Python output: 42' without running code is neither tool success nor grounds for execution reward.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Method comparison
How does outcome reward minus call cost compare with one fixed call per task?
Show answer and example
Answer: Fixed counts ignore difficulty and evidence, wasting calls on simple tasks and starving complex ones. Costed rewards adapt counts to value but can suppress needed tools if penalties are excessive. Report correctness and total call costs, not reductions alone.
Illustrative example: Example: Mental arithmetic can use zero calls, while multi-stage calculation may need two checks. One call does not fit both.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Basics
Why does ReTool learn calling, correction, and stopping together?
Show answer and example
Answer: Tools can help but fail and cost resources. A policy must decide when to call, how to respond to errors, and when evidence suffices. Tool syntax alone does not control a full trajectory. Early stopping and repeated unnecessary calls can both hurt.
Illustrative example: Example: Run code to check a calculation, fix arguments after an error, then stop once results agree rather than loop or submit unchecked.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why condition PinRec on desired outcomes and predict multiple item vectors for a future window?
Show answer and example
Answer: One next-item target overrepresents common behavior and imposes a unique order. Desired outcomes guide heads, and multiple ANN-compatible vectors cover coexisting interests. Align predictions with window positives; do not treat duplicate popular vectors as distinct interests.
Illustrative example: Illustration: Set a save goal rather than casual browsing, and cover both gardening and baking from an unordered window.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Why normalize PinRec ANN outputs per head and deduplicate items?
Show answer and example
Answer: Each vector must share index geometry and scale. Per-head normalization controls norms, and duplicate retrieved items need defined merging or deduplication. Match window target identities; repeating the same vector does not establish multi-interest learning.
Illustrative example: Illustration: Three heads returning one popular camera cover one item; count unique hits and inspect whether heads differ.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
Why are PinRec unordered-window retrieval and strict next-token hits different?
Show answer and example
Answer: Multiple window positives allow alternative orders and coexisting interests. Strict next-item metrics accept only the immediately adjacent target. Align labels and pools for comparison; both local measures were negative and a looser metric should not hide that.
Illustrative example: Illustration: Retrieving a camera and tent fits the day window, but a tent does not count as the next item if the camera came first.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Why is PinRec desired outcome different from the next observed feedback label?
Show answer and example
Answer: Desired outcome is a valid request condition. Future observed action is training supervision only; selecting conditions from test outcomes leaks information. Multi-vector windows represent several possible interests, not a guaranteed behavioral sequence.
Illustrative example: Illustration: Request save-worthy candidates, but do not first check that the held-out item was saved and choose the condition accordingly.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
Why repair critic quality and adapt GAE to length instead of dropping the critic?
Show answer and example
Answer: Sparse rewards and varying horizons amplify value error. VAPO pretrains the critic and adapts advantage estimation and actor to length. It doesn't remove value learning, and local proxy lengths aren't long-CoT evidence.
Illustrative example: Short and long solutions receive reward at different delays, so one estimator can have different bias/variance across them.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
Why isn't the local VAPO critic a reproduced pretrained value model?
Show answer and example
Answer: Candidate actions and proxy lengths simulate value–length coupling, not real long prefixes. Retain response-dependent GAE λ rather than claiming a fixed λ is equivalent.
Illustrative example: A table of candidate values isn't a neural critic reading thousands of reasoning tokens.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
What does VAPO trade for critic-free GRPO?
Show answer and example
Answer: GRPO uses same-prompt relative rewards without a critic. VAPO trains a critic for prefix values. Their costs and error sources differ; fewer models isn't higher capability.
Illustrative example: An all-wrong group has no relative signal while a critic can still estimate, possibly incorrectly.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
Why emphasize a critic for long reasoning in VAPO?
Show answer and example
Answer: Sparse terminal rewards leave prefixes without direct feedback; a critic supplies a baseline. Bias can grow with length, motivating pretrained value and length adaptation.
Illustrative example: Estimate later returns before an intermediate reasoning step gets terminal reward.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Core mechanism
Where does pixel shuffle put neighborhood information when it reduces visual token count?
SmolVLM: Redefining small and efficient multimodal models
Show answer and example
Answer: It concatenates nearby spatial tokens along channels and projects them through an MLP to the LM. It trades attention length for wider local representations rather than dropping patches. Fine text and spatial relations still require task-specific verification.
Illustrative example: Four adjacent patches merge into one wider vector, quartering token count but coarsening the LM's spatial granularity.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Implementation details
Does the local r=2 check include SmolVLM's high-resolution and long-video recipe?
SmolVLM: Redefining small and efficient multimodal models
Show answer and example
Answer: No. It uses 32×32 clothing images and 2×2 space-to-depth without high-resolution tiling, video sampling, long context, learned position tokens, or a SmolLM decoder. Results only check the rearrangement and projection.
Illustrative example: Reducing 16 patches to four verifies tensor shapes, not cross-frame localization or 8,000-token context.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Method comparison
What does space-to-depth preserve compared with averaging four patches?
SmolVLM: Redefining small and efficient multimodal models
Show answer and example
Answer: Averaging erases independent patch positions; space-to-depth concatenates them in fixed spatial order so later projection can distinguish them. It trades shorter sequences for wider input tokens. Fine recognition still depends on projector capacity and training.
Illustrative example: Swapping black upper-left and white lower-right blocks can preserve their average but change their ordered concatenation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Basics
Does SmolVLM just remove language-model layers?
SmolVLM: Redefining small and efficient multimodal models
Show answer and example
Answer: No. It reallocates vision and language compute with visual token compression, tiling, positional tokens, long context, and data mixtures. Parameter count and vision input cost are separate; one rearrangement operator is not the full recipe.
Illustrative example: A smaller LM still pays vision encoding and context costs if given many high-resolution visual tokens.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
Why isn't citation count enough when adding citation rewards to answer rewards?
Show answer and example
Answer: Citations connect claims to real evidence, while counts can reward repeated or irrelevant links. Search, browsing, and answering form one trajectory. Joint objectives should enforce support and correctness while penalizing wasted queries. Check claim–source links; one browse call does not establish evidence use.
Illustrative example: Ten irrelevant links should not beat two sources that directly support the conclusion, or the policy will learn link stuffing.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
How should DeepResearcher align citation rewards with actual retrieved evidence?
Show answer and example
Answer: Citations must point to genuinely available material supporting corresponding statements, not just count as links. N_query counts actual search calls. The mini-suite has no real web or full LLM RL, so fixture citations are not external evidence.
Illustrative example: A title never retrieved and irrelevant to the claim should not receive credit for looking like a citation.
Sections in the Chinese source: 核心公式, 复现边界. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
Why aren't citation quality rewards and query-count penalties the same efficiency objective?
Show answer and example
Answer: Citation reward measures evidential support; query penalty controls cost. Fewer searches do not ensure reliable evidence, and more citations do not ensure useful searches. The three weights govern trade-offs; short traces alone do not establish research quality.
Illustrative example: Answering quickly after one bad query is cheap but unsupported.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why does DeepResearcher treat search, browsing, evidence, and answers as one trajectory?
Show answer and example
Answer: Answers depend on how evidence is found and used, and final text alone obscures credit. Answer and citation rewards plus query cost balance quality and expense over the full search.
Illustrative example: One useful browse can make the answer verifiable, beyond making its final sentence fluent.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
Why remove both group SD and within-response averaging?
Show answer and example
Answer: SD reweights difficulty groups by their local variation; token averaging changes long-versus-short contribution. Keep centered rewards and fixed aggregation, not remove all normalization or promise universal gains.
Illustrative example: A tiny group SD can amplify a comparable reward gap; response length adds another weight.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
Can proxy lengths establish removal of long-CoT length bias?
Show answer and example
Answer: No. A candidate policy has no generated length distribution. Proxy lengths expose normalization differences, not real token losses or long responses.
Illustrative example: Changing a simulated length doesn't produce early stopping behavior.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
Does Dr. GRPO still center group rewards like GRPO?
Show answer and example
Answer: Yes. Subtract same-prompt mean, but omit the specified length and SD normalization. Dropping SD does not remove the baseline, and adding a global SD would change the target.
Illustrative example: Judge each reply relative to group mean without dividing by group variation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
Why audit both response-length averaging and group standard deviation in Dr. GRPO?
Show answer and example
Answer: They change weights by length and problem difficulty, not just numerical stability. Remove both while retaining group mean centering so reward differences preserve their scale.
Illustrative example: Equal reward gaps get different magnitudes after normalization when group variances differ.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
Why drop all-correct groups in Dynamic Sampling?
Show answer and example
Answer: Centering equal all-correct/all-wrong rewards yields no relative direction, not worthless replies. Resample differentiable groups; Clip-Higher, token aggregation and length shaping handle other effects. Never select questions on test.
Illustrative example: A mixed-success group provides group-relative learning absent from an equal-reward group.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
How does DAPO token-level normalization differ from per-response averages?
Show answer and example
Answer: Sum over valid response tokens, then normalize together. Per-response means weight lengths differently. Also preserve dynamic filtering and overlength penalty. Proxy lengths don't reproduce real masks or 32B training.
Illustrative example: Averaging a short and long reply separately is not averaging every token together.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
Which side does DAPO Clip-Higher relax?
Show answer and example
Answer: Raise the upper bound to retain room for beneficial probability increases; lower bound remains separate. Dynamic sampling, token loss and overlength shaping are also defining pieces.
Illustrative example: Easier increase of a useful action needn't loosen downward changes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
Why dynamically skip all-correct and all-wrong DAPO groups?
Show answer and example
Answer: Binary rewards without variance give no relative advantage and waste batch slots. Sample groups with distinctions; this doesn't solve exploration on every difficult problem.
Illustrative example: Four failures give no ordering, so resample rather than repeatedly train on zeros.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
Why mask retrieved tokens even though they occur in the input sequence?
Show answer and example
Answer: Search returns are environment output, not sampled policy actions. Policy loss there would teach the model to generate external evidence. Only its queries, reasoning, and answers are policy tokens; retrieval conditions later decisions. Iterative adaptation is not equivalent to a single static RAG retrieval.
Illustrative example: A search result that says 'published Tuesday' is not agent behavior. Train how to query and answer from evidence, not how to generate that environment text.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
How does Search-R1's loss mask avoid masking the query action?
Show answer and example
Answer: Separate model-generated reasoning and search instructions from environment information. The former remain policy tokens and the latter get mask zero. Use provenance boundaries, not a text match for 'search.'
Illustrative example: The query inside `<search>` is an action; the following `<information>` text is an observation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
How does visible search context differ from training on retrieved text as policy-generated tokens?
Show answer and example
Answer: Documents can condition later actions, but their tokens come from the environment, not the policy. Mask their loss. Treating them as student outputs corrupts action probabilities for policy optimization.
Illustrative example: A thousand-character webpage does not count as a thousand-character agent action with policy gradients.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why does Search-R1 repeatedly decide whether to search rather than using one-shot RAG?
Show answer and example
Answer: Intermediate evidence can expose questions or revise a belief, requiring another query. Search is an environment; reasoning and search alternate, and final answer outcomes train the policy.
Illustrative example: A first retrieval reveals entity ambiguity, so the agent can query disambiguating conditions before answering.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Core mechanism
Why can correct final database state still fail SOPBench?
SOPBench: Evaluating Language Agents at Following Standard Operating Procedures and Constraints
Show answer and example
Answer: The service must be permitted and required helpers executed in order. And, Or, and Chain encode distinct conditions; guessing the result cannot bypass procedure. Full success requires eligibility, state, and procedure together.
Illustrative example: A refund amount can be correct while the agent skipped order eligibility. A later check does not satisfy a chain requiring eligibility first.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Implementation details
Can agent self-reports replace SOPBench helper-call records?
SOPBench: Evaluating Language Agents at Following Standard Operating Procedures and Constraints
Show answer and example
Answer: No. procedure_complete needs trusted execution traces showing actual checks in graph order. Gold stays with graders. This protocol component is not a full production service agent.
Illustrative example: Saying 'I verified it' is not a logged successful helper call.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Method comparison
How do And, Or, and Chain constraints differ in verification order?
SOPBench: Evaluating Language Agents at Following Standard Operating Procedures and Constraints
Show answer and example
Answer: And needs every condition without imposing suborder. Or allows a verified true branch. Chain enforces sequence. Treating them as unordered sets permits violations like acting before checking.
Illustrative example: Identity verification before a transfer is Chain; checking after the transfer does not satisfy it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Basics
Do SOPBench service constraint nodes record chat turns or define required pre-service checks?
SOPBench: Evaluating Language Agents at Following Standard Operating Procedures and Constraints
Show answer and example
Answer: The graph connects services to constraints and constraints to executable helper checks, specifying permission and prerequisites rather than summarizing dialogue order. Follow each service's dependencies to see what must be verified, not just whether final text sounds fluent.
Illustrative example: If a service requires eligibility, the graph should link to that constraint and helper rather than merely contain 'I checked.'
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why does COBRA generate a dense vector conditioned on an already generated sparse SID?
Sparse Meets Dense: Unified Generative Recommendations with Cascaded Sparse-Dense Representations
Show answer and example
Answer: Quantization retains coarse structure but loses within-code details. The discrete path narrows search; user context and that path generate a dense query to distinguish items sharing a coarse code. Joint objectives align stages, but reranking cannot guarantee recovery of paths eliminated by the beam.
Illustrative example: After choosing athletic shoes, a dense query can distinguish trail from road shoes. It cannot recover them if the beam picked appliances.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Why combine COBRA beam scores and conditioned-vector dot products at inference?
Sparse Meets Dense: Unified Generative Recommendations with Cascaded Sparse-Dense Representations
Show answer and example
Answer: Dot products express item fit conditional on a code; beam scores preserve each path's prior plausibility. Combine on compatible scales and choose coefficients on validation. The dense training loss should actually update conditioned vectors, not be replaced by a fixed score mix.
Illustrative example: A high local dot product under an unlikely code should not automatically beat a slightly lower score under a likely code.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
What does COBRA borrow from continuous two-tower retrieval and pure SID generation?
Sparse Meets Dense: Unified Generative Recommendations with Cascaded Sparse-Dense Representations
Show answer and example
Answer: Continuous towers suit ANN but often compress a user into few vectors; SID generation nominates hierarchical regions at the cost of quantization errors. COBRA uses codes for multi-region nomination and conditioned vectors for within-region detail, adding modules and calibration. Sparse-only ablation and comparison with DIN answer different questions.
Illustrative example: A beam can nominate photography and baking separately and refine both, instead of averaging interests in one vector, but both score levels need maintenance.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Why generate COBRA's sparse code and dense vector in separate stages?
Sparse Meets Dense: Unified Generative Recommendations with Cascaded Sparse-Dense Representations
Show answer and example
Answer: A beam selects semantic regions by discrete code, then conditioned nearest-neighbor search preserves fine differences within each region and offsets quantization loss. Dense precision cannot recover items outside every selected region.
Illustrative example: Locate science fiction first, then distinguish hard and action science fiction. Fine ranking within family comedy will not find a science-fiction target.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why train OneRec preference pairs from full current-policy sessions rather than individual items?
OneRec: Unifying Retrieve and Rank with Generative Recommender and Iterative Preference Alignment
Show answer and example
Answer: Session-level generation and reward can model coherence and diversity. Self-hard winner/loser pairs focus DPO on complete outputs the model confuses, comparing probability changes against a reference. Reward errors can steer updates incorrectly, and local results do not validate online gains.
Illustrative example: Illustration: Two five-video slates can be individually relevant but differ in repetition and flow. Session preferences teach which items work together.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Why don't legal SID prefix constraints solve semantic-collision identity problems in OneRec?
OneRec: Unifying Retrieve and Rank with Generative Recommender and Iterative Preference Alignment
Show answer and example
Answer: A legal prefix proves catalog existence, not a unique item. Collisions require defined expansion or disambiguation, and uniqueness affects evaluation. Generate winners and losers from the current policy without gold injection. Legality, rewards, and ranking are separate checks; retain negative results.
Illustrative example: Illustration: Two films can share one legal code. Expanding to the correct film by reading the test label leaks the answer.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
Does OneRec self-hard DPO change the model or the search compared with widening the beam?
OneRec: Unifying Retrieve and Rank with Generative Recommender and Iterative Preference Alignment
Show answer and example
Answer: A wider beam explores the existing distribution; DPO updates policy probabilities using current-policy session pairs. Weak reward judging can hurt: local DPO loss fell quickly while NDCG went to zero. Preference fit does not establish ranking quality.
Illustrative example: Illustration: Search may find a good list, but training on the judge's wrong winners can worsen future distributions despite a lower loss.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Why concatenate multiple item SIDs into one autoregressive OneRec session target?
OneRec: Unifying Retrieve and Rank with Generative Recommender and Iterative Preference Alignment
Show answer and example
Answer: Later items can condition on earlier selections, unlike independent item generation. Hierarchical SIDs give composable addresses and reward evaluates the session. Order still matters, and diversity is not guaranteed.
Illustrative example: Illustration: Follow a beginner photography tutorial with advanced practice instead of generating three independently relevant beginner videos.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why does FilterLLM predict an audience distribution from item text rather than asking an LLM about each user?
FilterLLM: Text-To-Distribution LLM for Billion-Scale Cold-Start Recommendation
Show answer and example
Answer: Pairwise decisions scale with audience size and are costly for new items. Encode the item once and predict probabilities over a compressed user vocabulary supervised by historical behavior. Compression trades sharing for resolution; group probabilities are not reliable personalized judgments for each member.
Illustrative example: Predict an audience distribution once for a new article instead of generating one LLM answer per user, while accepting loss of individual detail.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
Why not equate nonexposure with dislike in FilterLLM target distributions?
FilterLLM: Text-To-Distribution LLM for Billion-Scale Cold-Start Recommendation
Show answer and example
Answer: Observed interactions reflect exposure. Specify `q(u|i)` normalization, vocabulary mapping, and training cutoff; test interactions cannot build targets. Otherwise the model copies prior exposure or leaks future evidence.
Illustrative example: A user who never saw a tent might like camping; forcing zero for all unexposed users prevents finding new audiences.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
How does a compressed user vocabulary trade output size against personalization versus per-user classification?
FilterLLM: Text-To-Distribution LLM for Billion-Scale Cold-Start Recommendation
Show answer and example
Answer: Clustering users produces a smaller distribution covering many individuals but loses within-group detail and changes with data and grouping. The local genre-text proxy regressed, so scalability alone is not cold-start quality evidence.
Illustrative example: A highly scored beginner photography group includes someone who just bought a camera and may not need another.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why predict audiences rather than next items from new product text in FilterLLM?
FilterLLM: Text-To-Distribution LLM for Billion-Scale Cold-Start Recommendation
Show answer and example
Answer: Cold-start items have little interaction, so the task is to find likely users proactively. One encoding yields a compressed audience distribution mapped back to people. Scores are modeled inclinations, not observed reactions or explicit rejection.
Illustrative example: Estimate interest among camping and travel audiences for a new tent without a billion individual LLM calls, then collect real feedback.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Core mechanism
Why treat gradients as matrices rather than adjust coordinates like AdamW?
Muon is Scalable for LLM Training
Show answer and example
Answer: Muon approximates the polar direction of momentum for hidden matrices, then applies shape scaling and decay. This uses spectral relationships, not just elementwise moments. Non-hidden parameters still use AdamW in the large recipe.
Illustrative example: Nearly collinear columns can yield different directions under matrix polar transformation and coordinate scaling.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Implementation details
Why divide Newton–Schulz input by its Frobenius norm?
Muon is Scalable for LLM Training
Show answer and example
Answer: Normalize iteration scale before the specified polynomial; handle zero or tiny norms and preserve shape scaling. Orthogonality alone does not establish optimization quality or runtime.
Illustrative example: Scaling a gradient uniformly should not radically change the iteration's numeric range.
Sections in the Chinese source: 核心公式, 本地复现. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Method comparison
How does local Muon differ from applying it to every parameter?
Muon is Scalable for LLM Training
Show answer and example
Answer: Hidden matrices use Muon; embeddings, norms, biases, heads, and other parameters use AdamW. The hybrid optimizer is part of the comparison.
Illustrative example: A one-dimensional norm parameter is not suitable for matrix Newton–Schulz.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Basics
Why is Muon a matrix geometry update?
Muon is Scalable for LLM Training
Show answer and example
Answer: Hidden weights form linear maps. Approximate polar decomposition acts on momentum jointly and uses shape scaling and decay.
Illustrative example: A dominant correlated direction is handled differently from independent scalar learning rates.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Core mechanism
Why add local self-distillation and masked prediction to pairwise sigmoid alignment?
Show answer and example
Answer: Global matching constrains whole-image compatibility with text but need not train local or dense features. Local distillation and masking supervise spatial detail; captioning and data selection improve semantic inputs. Ablate these separately rather than credit sigmoid alone.
Illustrative example: A street caption may only mention a car, while recovery around its masked wheel encourages finer spatial features.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Implementation details
How do label signs and bias enter sigmoid pair loss?
Show answer and example
Answer: Use z=+1 or −1 in log(1+exp(−z·score)), with scaled dot product plus bias as score. Substituting 0/1 without changing the formula removes the negative gradient. Local work covers class text and masked-view consistency only.
Illustrative example: z=−1 penalizes high negative-pair similarity; z=0 makes loss constant.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Method comparison
How does pairwise sigmoid differ from CLIP batch softmax?
Show answer and example
Answer: Softmax normalizes over each row or column so a matching pair competes with other batch entries. Sigmoid gives each image–text pair an independent positive or negative classification term. Normalization, temperature, and bias differ; effectiveness is empirical.
Illustrative example: Cat image–cat text is positive and cat image–dog text negative; sigmoid applies two separate binary constraints.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Basics
Does SigLIP 2 NaFlex change the number of class labels?
Show answer and example
Answer: No. Dynamic resolution changes how vision input is represented and where compute goes. Labels are separate. SigLIP 2 combines input adaptation, global semantics, and local feature training; a fixed-resolution loss kernel does not cover the complete recipe.
Illustrative example: The same cat and dog labels can accompany images of different sizes.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why use human and LLM CDI preferences with IPO rather than maximize logged hits?
Bursting Filter Bubble: Enhancing Serendipity Recommendations with Aligned Large Language Models
Show answer and example
Answer: Logs favor past exposures. Layered profiles supply context, CDI adds judgments beyond logs, IPO aligns preference pairs with SFT constraints, and nearline caching lowers serving cost. Validate relevance and novelty separately.
Illustrative example: Compare a suitable unseen film against a familiar one; logged hits alone favor repetition.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
Why generate and cache nearline rather than per request?
Bursting Filter Bubble: Enhancing Serendipity Recommendations with Aligned Large Language Models
Show answer and example
Answer: Move expensive generation off the critical path and let ranking consume cached candidates. Version profiles/cache and refresh stale candidates. Local work is not production inventory or full human CDI.
Illustrative example: Refresh after interests change instead of reusing old candidates forever.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
Is SERAL fully better if hit rate rises but novelty falls?
Bursting Filter Bubble: Enhancing Serendipity Recommendations with Aligned Large Language Models
Show answer and example
Answer: No. Serendipity needs relevance and unseen content. Report both and their trade-off; offline proxies do not establish real user surprise or conversions.
Illustrative example: Familiar popular items can improve hits while offering fewer unseen items.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why define serendipity as relevant and unseen?
Bursting Filter Bubble: Enhancing Serendipity Recommendations with Aligned Large Language Models
Show answer and example
Answer: Novelty alone can be irrelevant and relevance alone can repeat a feedback loop. Layered profiles and preference controls seek both before nearline candidates enter ranking. Novelty does not guarantee satisfaction.
Illustrative example: Recommend unfamiliar content fitting a persistent interest, not an arbitrary obscure item.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Core mechanism
Why keep the current causal block when MoBA routes to a few remote ones?
MoBA: Mixture of Block Attention for Long-Context LLMs
Show answer and example
Answer: Routing finds distant evidence; current-block retention protects nearby context from routing misses. Causal masking restricts visible positions, and attention inside selected blocks is exact at token level. Sparsity does not preserve every distribution.
Illustrative example: Keep the current subject even if remote chapters score higher; do not read the current block's ungenerated suffix.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Implementation details
Does selecting the current MoBA block remove the need for causality inside it?
MoBA: Mixture of Block Attention for Long-Context LLMs
Show answer and example
Answer: No. Mask positions after the query inside the selected block. The local eight-token, top-two configuration is not a million-context fused kernel.
Illustrative example: Word three cannot read word seven merely because both share an eight-token block.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Method comparison
What can content-routed MoBA recover that a recent window cannot?
MoBA: Mixture of Block Attention for Long-Context LLMs
Show answer and example
Answer: A window excludes positions by age. Routing can recover relevant distant blocks but can still miss evidence. Cost depends on block size and routing budget; it is not lossless pruning.
Illustrative example: A name at the beginning can return through content matching instead of being automatically excluded.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Basics
Why group history into blocks instead of reading all of it for every query?
MoBA: Mixture of Block Attention for Long-Context LLMs
Show answer and example
Answer: Routing picks a few relevant blocks, then exact attention reads inside them. This reduces candidate pairs while keeping within-block detail.
Illustrative example: A question needing two passages need not attend to every word.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Core mechanism
Why retain original blocks and a sliding window in addition to compressed NSA history?
Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention
Show answer and example
Answer: Compression gives cheap global context but can lose details. Query-selected original blocks recover precise evidence, and a window covers recent tokens. All three paths learn and fuse from pretraining and remain causal.
Illustrative example: A summary can identify the chapter about dates; exact selected tokens resolve the year and the window preserves current qualifiers.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Implementation details
Do all three NSA branches need causal boundaries?
Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention
Show answer and example
Answer: Yes. Compressed blocks, selected fine blocks, and windows cannot expose future information. Align output dimensions before gated fusion. Local edge counts do not measure paper Triton throughput.
Illustrative example: Mask future words in a selected block spanning the query position.
Sections in the Chinese source: 核心公式, 复现边界. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Method comparison
How does NSA differ from pruning low attention weights after training?
Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention
Show answer and example
Answer: Three paths and their gates are trained jointly from pretraining, changing what the model learns to access. Post-hoc deletion from a dense model does not provide the same adaptation.
Illustrative example: The model learns to trade summary and detailed blocks rather than suddenly losing history.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Basics
Why use compression, selection, and window paths together?
Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention
Show answer and example
Answer: They provide global overview, exact selected detail, and exact recent context, respectively. Learned gates combine them; one sparsity pattern need not serve all roles.
Illustrative example: Get topic summary, a distant key sentence, and current phrase detail from different paths.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why aggregate sessions before predicting multiple next-session positives?
SessionRec: Next Session Prediction Paradigm For Generative Sequential Recommendation
Show answer and example
Answer: Items within a visit often share intent rather than a unique order. ISE aggregates; SSE learns cross-visit transitions. Supervise all next-session positives and use actual skipped impressions. Bad session boundaries can mix intents.
Illustrative example: Group tent, bag, and mat as one shopping visit before predicting the next hiking visit.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Can random catalog negatives be labeled skipped exposures?
SessionRec: Next Session Prediction Paradigm For Generative Sequential Recommendation
Show answer and example
Answer: No. Hard negatives are actual exposed but unclicked items in that visit. Training samples a positive per round for CE, while evaluation keeps the full positive set.
Illustrative example: An unseen movie is not a rejected impression.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
Which ordering do ISE and SSE handle?
SessionRec: Next Session Prediction Paradigm For Generative Sequential Recommendation
Show answer and example
Answer: ISE aggregates items within a visit, and SSE models chronological visit transitions. Mean ISE downplays within-visit order, not all order. SSE can use Transformer or HSTU.
Illustrative example: Noon versus evening visits matter even if within-visit browse ordering is weak.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Is SessionRec's session token one item or a visit representation?
SessionRec: Next Session Prediction Paradigm For Generative Sequential Recommendation
Show answer and example
Answer: ISE aggregates multiple items into one visit representation; SSE models transitions across visits. The next visit can include several positives.
Illustrative example: Group A/B/C into one visit and target D/E together.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why place LUM's condition before the item being predicted rather than after the previous item as feedback?
Show answer and example
Answer: The condition specifies the upcoming choice's context, query, or category and should constrain the next-item distribution. Attaching it to the previous item instead explains past behavior. LUM learns queryable user knowledge through conditional generation and extracts offline interest features for a DLRM rather than serving the full generator.
Illustrative example: Illustration: Entering kitchenware should condition the next cookware prediction, not be interpreted as feedback on the previous running shoe.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
Why mask query-to-query attention when appending several LUM conditions offline?
Show answer and example
Answer: Each condition should independently read the same shared history. Mutual attention makes results depend on query order and breaks reuse. Predict the immediately following item from the condition output, not from an action after the item. Bind caches to the same prefix and condition version.
Illustrative example: Illustration: High- and low-rating queries should not exchange clues; reversing their order must not change the knowledge each retrieves.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
How do LUM's multiple conditional queries organize information differently from a fixed user embedding?
Show answer and example
Answer: One vector compresses all needs; conditional queries can extract distinct contexts or preferences for the downstream target. More queries require cache space and can be redundant. Gains over a sequence-free ID-DLRM include several new information sources and cannot be assigned solely to condition tokens.
Illustrative example: Illustration: High-rating and low-rating queries may reveal likes and dislikes separately, while an average can mix them together.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why does LUM separate user-knowledge generation and online DLRM ranking into three stages?
Show answer and example
Answer: First train a generative model on conditional user relationships; then query and cache different interests offline; finally supply knowledge vectors and target-match scores to the existing discriminative ranker. This avoids serving the large model on every request. Transfer through a DLRM interface is not necessarily lossless, and freshness and condition definitions matter.
Illustrative example: Illustration: Prepare multiple interest profiles offline and let the online DLRM match them to a product without replacing its real-time feature system.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why does FuXi-α use multiple interaction channels and staged FFNs instead of widening one attention path?
FuXi-α: Scaling Recommendation Model with Feature Interaction Enhanced Transformer
Show answer and example
Answer: One path can mix unlike relationships in the same space. Adaptive channels model different interactions and multistage FFNs combine them gradually. Additional channels can still learn redundant patterns; compare single-channel and matched-budget configurations.
Illustrative example: User–item, context–item, and history–candidate relations can be learned separately, but identical attention channels only add cost.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
What shapes and update order does FuXi-α require for progressive residual fusion?
FuXi-α: Scaling Recommendation Model with Feature Interaction Enhanced Transformer
Show answer and example
Answer: Channels may have distinct Q/K/V. Project their concatenated state back to backbone width for residual addition, preserve layer and old/new state order, and train channel weights rather than concatenate fixed similarities. Local comparisons only cover the implemented budget.
Illustrative example: Three d-wide channel outputs plus backbone features must pass through an FFN back to d before addition.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
Where does FuXi-α's staged fusion differ from concatenating channel outputs only at the final head?
FuXi-α: Scaling Recommendation Model with Feature Interaction Enhanced Transformer
Show answer and example
Answer: Intermediate FFNs fuse current state and channel outputs before later layers, allowing further relation composition. Late fusion has one opportunity. Additional capacity costs more and did not prevent local NDCG decline.
Illustrative example: A later layer can combine nighttime context and price sensitivity into demand for evening promotions rather than only merging final summaries.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why divide feature interactions across FuXi-α channels?
FuXi-α: Scaling Recommendation Model with Feature Interaction Enhanced Transformer
Show answer and example
Answer: Heterogeneous fields encode different relations; widening one space can still favor the strongest. Separate adaptive channels and staged FFN fusion can retain complementary structure, not guarantee it.
Illustrative example: Channels can specialize in user–item fit and context–price interaction, or all collapse to popularity.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Core mechanism
Why exclude the current trajectory from its leave-one-out baseline?
Show answer and example
Answer: Other group returns supply a critic-free reference without using the trajectory's own reward in it. Long rollouts are expensive, making token-level ratio clipping and batch reuse useful, but repeated updates still face policy drift. Old trajectories do not become indefinitely fresh on-policy data.
Illustrative example: For rewards one, one, and zero, compare the first with the mean of the other two, then clip locally drifting tokens separately.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Implementation details
Can LOOP compute its leave-one-out baseline directly when K=1?
Show answer and example
Answer: No. K−1 is zero, so a singleton fails the group definition and needs explicit handling or rejection. Multi-epoch reuse must preserve the corresponding old policy probabilities and episode identity, not reinsert the sample's own return into its baseline.
Illustrative example: With one example, the mean of other examples does not exist.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Method comparison
How does LOOP token-level clipping differ from discarding a trajectory as soon as one token drifts?
Show answer and example
Answer: Local clipping constrains drift while retaining other useful parts of a long trace. Whole-trajectory dropping loses more expensive interaction data. Reuse must preserve old behavior probabilities; off-policy samples are not newly on-policy.
Illustrative example: One abnormal ratio in a hundred-step trajectory need not erase all other feedback.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Interactive policy learning: converting trajectory feedback into updates
Basics
Why does LOOP use several same-task trajectories instead of an independent value network?
Show answer and example
Answer: Their returns provide a leave-one-out baseline without maintaining a critic. A PPO trust region supports reuse of expensive rollouts. Removing a value network does not remove importance ratios or drift control.
Illustrative example: Compare one trajectory against the mean return of the other traces for its task.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why follow multimodal masked reconstruction with content-interest-aware SFT in MIM?
MIM: Multi-modal Content Interest Modeling Paradigm for User Behavior Modeling
Show answer and example
Answer: Reconstruction learns internal content structure but not which differences users care about. SFT aligns semantic features with consumption interests, then content and ID interest fuse to combine cold-start and collaborative signals. Reconstruction alone does not establish ranking relevance, and either branch can overwhelm the other.
Illustrative example: Illustration: Reconstructing product color does not show whether users care about lightness or durability. Interest alignment introduces consumption evidence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
Why use different visibility rules for MIM masked reconstruction and recommendation targets?
MIM: Multi-modal Content Interest Modeling Paradigm for User Behavior Modeling
Show answer and example
Answer: Training can mask content and reconstruct it, but recommendation input stops at the current history cutoff. Future items and feedback are targets only. Modality masking cannot fix a future item already present in the user representation.
Illustrative example: Illustration: Masking part of a historical poster is valid content training; putting the test film's full vector in history before predicting it leaks the target.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
What cold-start clues does MIM's dual-interest fusion add beyond ID co-occurrence?
MIM: Multi-modal Content Interest Modeling Paradigm for User Behavior Modeling
Show answer and example
Answer: ID co-occurrence needs feedback; shared multimodal semantics can transfer to sparse items. Content noise and bad mappings can hurt ranking. Local NDCG rose slightly while Hit fell, so report both and evaluate a dedicated new-item cohort.
Illustrative example: Illustration: An unseen movie may be identified as science fiction from content, but a misleading action poster can also hurt otherwise sound ID scores.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why align content with interest after multimodal reconstruction in MIM?
MIM: Multi-modal Content Interest Modeling Paradigm for User Behavior Modeling
Show answer and example
Answer: Reconstruction teaches recoverable features, not their relation to user preference. Content-interest-aware SFT links semantic and collaborative information; CiUBM combines content and ID interests. Low reconstruction loss cannot substitute for ranking evidence.
Illustrative example: Illustration: Recovering poster colors does not identify a preference for hard science fiction over comedy without behavioral supervision.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
How do AdaF²M² multiple masked forwards differ from dropping features once?
AdaF²M²: Comprehensive Learning and Responsive Leveraging Features in Recommendation System
Show answer and example
Answer: Several masks retain complementary views. A state-aware adapter combines them by user/item reliability; one perturbed output or uniform averaging does not implement this.
Illustrative example: Combine behavior-masked and content-masked views differently for new and mature items.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Which axis should AdaF²M² softmax normalize?
AdaF²M²: Comprehensive Learning and Responsive Leveraging Features in Recommendation System
Show answer and example
Answer: Normalize branches separately for each example, with compatible representation widths. Do not couple different users through a batch-level softmax.
Illustrative example: Two users with three views each need two independent three-weight sums.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does multi-forward fusion differ from feature dropout?
AdaF²M²: Comprehensive Learning and Responsive Leveraging Features in Recommendation System
Show answer and example
Answer: Single dropout trains robustness to missing inputs. Multi-forward retains alternatives and learns state-specific fusion, adding execution cost even at equal parameter count.
Illustrative example: Keeping price-present and price-absent predictions for fusion differs from randomly deleting price once.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why does state-aware fusion vary across examples?
AdaF²M²: Comprehensive Learning and Responsive Leveraging Features in Recommendation System
Show answer and example
Answer: Activity and feedback change which signals are trustworthy. The adapter weights complementary masks per state; one fusion coefficient is not permanent importance.
Illustrative example: Behavioral history can dominate for an established user and content for a new one.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
Why center locally but scale globally?
Show answer and example
Answer: Group mean removes common difficulty; local SD may amplify incidental low variance. Cross-batch scale makes units consistent without critic. Window and distribution matter; small EMA tests don't prove stability.
Illustrative example: Two similarly separated groups shouldn't receive very different weights just because one has tiny variance.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
Can REINFORCE++ reestimate EMA from evaluation gold?
Show answer and example
Answer: No. Maintain training state in batch order; test answers cannot tune scale. Small-candidate EMA validates semantics, not the large-batch distribution.
Illustrative example: Scanning all test rewards to set the denominator leaks evaluation information.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
Why isn't global EMA normalization a reordering of GRPO normalization?
Show answer and example
Answer: Statistics use different domains. Group means compare same-prompt samples; global second moments share scale. This changes cross-problem weighting.
Illustrative example: Groups with the same gap but different variances differ under local division, not a shared scale.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
Why avoid each prompt's SD in REINFORCE++?
Show answer and example
Answer: Few samples make local variance noisy and can overamplify some difficulty groups. Keep group centering and use a global scale without a critic.
Illustrative example: An accidentally tiny variance should not give one prompt excessive weight.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Core mechanism
Why doesn't BLT pay equal global compute for every byte?
Byte Latent Transformer: Patches Scale Better Than Tokens
Show answer and example
Answer: Local modules handle bytes, entropy sets variable patch boundaries, and global attention operates on patches before local byte decoding. Easy spans can use longer patches. This saves global interactions, not byte handling, and is not fixed tokenization.
Illustrative example: Repeated patterns can produce fewer global patches than equally long unpredictable bytes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Implementation details
Why isn't the local surprisal boundary path full BLT?
Byte Latent Transformer: Patches Scale Better Than Tokens
Show answer and example
Answer: A 512-symbol tokenizer approximates bytes with learned boundaries and shared latents, but does not implement full local encoders/decoders or 8B training. Do not label the approximate alphabet raw bytes.
Illustrative example: Boundary-aware grouping alone does not supply the complete byte-to-byte architecture.
Sections in the Chinese source: 复现边界. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Method comparison
How do BLT boundaries differ from fixed tokenizer boundaries?
Byte Latent Transformer: Patches Scale Better Than Tokens
Show answer and example
Answer: A fixed tokenizer uses predefined subword rules. BLT uses prefix-dependent next-byte entropy, changing compute granularity by uncertainty.
Illustrative example: The same character sequence may get different boundaries in different contexts.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Basics
Why put patches between bytes and global attention?
Byte Latent Transformer: Patches Scale Better Than Tokens
Show answer and example
Answer: Per-byte global compute is costly. Next-byte uncertainty forms variable patches and local encoders/decoders bridge them to byte inputs and outputs.
Illustrative example: Predictable bytes can group more densely than difficult spans.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why train IDs with frozen text representations before joint unfreezing in PRECISE?
PRECISE: Pre-training Sequential Recommenders with Collaborative and Semantic Information
Show answer and example
Answer: Strong pretrained text can dominate initially random IDs and leave collaborative structure unlearned. The frozen stage lets IDs learn distinctions before joint adaptation. All-scene pretraining and sparse-target tuning address transfer separately; overall gains do not guarantee cold or tail gains.
Illustrative example: Illustration: Two similar movie descriptions can attract different audiences; learning IDs first can capture that difference before fusion.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
Why must PRECISE still exclude target items when switching from causal UT to noncausal TT?
PRECISE: Pre-training Sequential Recommenders with Collaborative and Semantic Information
Show answer and example
Answer: TT can read observed history bidirectionally, not the held-out target. Remove the target before constructing the sequence; the concatenation MLP sees legal positions only. Repeated in-batch items must not be treated as mutually exclusive negatives, as fixed in source.
Illustrative example: Illustration: Historical A and B can attend to one another, while test C remains outside the input.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
What details does PRECISE token-attention MoE retain compared with averaging text?
PRECISE: Pre-training Sequential Recommenders with Collaborative and Semantic Information
Show answer and example
Answer: Token experts can focus on distinct spans and top-k routing can combine perspectives, while averaging flattens them. Experts can duplicate one another and strong text can suppress IDs, motivating the frozen-text stage. Validate complementarity and cold/tail outcomes.
Illustrative example: Illustration: Different experts can preserve 'beginner' versus 'professional'; experts all reading celebrity names do not retain meaningful detail.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why do PRECISE UT and TT address all-scenario interests and sparse targets separately?
PRECISE: Pre-training Sequential Recommenders with Collaborative and Semantic Information
Show answer and example
Answer: Broad sequence training supplies base preferences and target training adapts scarce labels. Objectives and attention rules differ, and transfer can hurt cold/tail items. Overall local gains accompanied lower Cold Recall, not universal cold-start improvement.
Illustrative example: Illustration: Learn interests from all positive feedback and then tune on high ratings. Popularity may improve aggregate hits while missing new films.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why distill teacher keywords and reasoning and then align the student to CTR in MSD rather than cache teacher embeddings alone?
Balancing Efficiency and Effectiveness: An LLM-Infused Approach for Optimized CTR Prediction
Show answer and example
Answer: Static embeddings save compute but do not adapt to CTR. Autoregressive distillation transfers teacher semantics; LoRA and a recommendation adapter learn task features. Cache frequent items, encode misses online, and fuse only relevant history. These address semantics, CTR, and serving separately.
Illustrative example: Illustration: A student learns coat warmth and commuting attributes from teacher text, adapts them using clicks, then serves cached popular coats and encodes new items online.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
Why separately check candidate-specific top-three history and shareable cache features in MSD?
Balancing Efficiency and Effectiveness: An LLM-Infused Approach for Optimized CTR Prediction
Show answer and example
Answer: History selection depends on target similarity and can differ per candidate; cached item semantics must not include user or candidate-pair interaction. Verify only designated LoRA parameters update in the frozen student. Mixing paths can contaminate caches or erase candidate relevance.
Illustrative example: Illustration: Cameras need photography history and shoes need running history. Their selected top-three sets cannot be shared even if base item encodings can.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
How does MSD amortize compute with frequent-item caching and online encoding for rare items?
Balancing Efficiency and Effectiveness: An LLM-Infused Approach for Optimized CTR Prediction
Show answer and example
Answer: Frequent items yield cache reuse; online encoding preserves coverage for rare items and misses. Encoding everything online is fresher but repeats expensive work. Stale cache/LoRA versions, refreshes, and hit rate matter. The local path does not measure production embedding efficiency.
Illustrative example: Illustration: Cache a coat scored a million times a day, encode a new raincoat online, and refresh cached semantics when LoRA changes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why split MSD's MKDM and MKIM into knowledge distillation and CTR adaptation?
Balancing Efficiency and Effectiveness: An LLM-Infused Approach for Optimized CTR Prediction
Show answer and example
Answer: Teacher keyword and reasoning generation transfers semantics first. LoRA and the recommendation adapter then align those features to clicks, unlike frozen knowledge. Supervision and trainable parameters differ across stages; lower distillation loss does not guarantee higher CTR.
Illustrative example: Illustration: First learn what warmth and commuting mean for coats, then train how user clicks combine them.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why train LEADRE ad SID generation with an intent prompt, semantic auxiliary tasks, and DPO?
LEADRE: Multi-Faceted Knowledge Enhanced LLM Empowered Display Advertisement Recommender System
Show answer and example
Answer: Predicting only discrete codes can memorize exposed IDs instead of using ad content and cross-domain needs. The intent prompt organizes long-term and real-time signals; semantic and commercial value objectives make codes more useful; DPO adjusts preference trade-offs. Report SFT-to-DPO changes separately because preference tuning need not improve every retrieval metric.
Illustrative example: Illustration: Two appliance ads may differ in relevance to current renovation, even if one appears much more often. Semantic and value supervision can separate them, but too much DPO weight on commercial feedback may hurt new relevant candidates.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Why must LEADRE keep the reference and context aligned when constructing winning and losing ad SID paths?
LEADRE: Multi-Faceted Knowledge Enhanced LLM Empowered Display Advertisement Recommender System
Show answer and example
Answer: The objective compares winner and loser probability changes relative to a reference under the same user intent. Switching requests or references changes the target. Construct legal SID mappings and auxiliary labels at the cutoff, do not use test answers in prompts, and version long-term caches with real-time inputs.
Illustrative example: Illustration: Two legal SID paths preferred by user A must not be scored against a reference conditioned on user B's history.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
How can LEADRE's full system improve while its DPO ablation worsens?
LEADRE: Multi-Faceted Knowledge Enhanced LLM Empowered Display Advertisement Recommender System
Show answer and example
Answer: The full system changes SID and semantic/value auxiliary tasks, whereas the ablation adds preference updates to the same SFT. Beating DIN does not validate each change. The local DPO result regressed relative to SFT, so do not attribute the full gain to alignment.
Illustrative example: Illustration: Pretraining may deliver a large gain, DPO may give some back, and the final system still beats a weaker baseline.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Why combine cross-domain, long-term, and real-time behavior in LEADRE's intent-aware prompt?
LEADRE: Multi-Faceted Knowledge Enhanced LLM Empowered Display Advertisement Recommender System
Show answer and example
Answer: Cross-domain activity supplies clues unavailable in ads, long-term summaries retain stable interests, and real-time behavior reflects change. Together they help S-ID generation do more than memorize IDs. Summaries and caches can be stale and sources may conflict; a longer prompt is not full user understanding.
Illustrative example: Illustration: A stable photography preference, recent travel search, and current camera browsing suggest portable gear even if the new ad has no in-domain click history.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Core mechanism
Why put attention and SSM in parallel within Hymba layers?
Hymba: A Hybrid-head Architecture for Small Language Models
Show answer and example
Answer: The same input gets direct historical reads and a recurrent summary in one layer, then fuses both. This combines access mechanisms, not free compute; fusion, KV, and state budgets still matter.
Illustrative example: One branch retrieves an old name while the other maintains running grammar state.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Implementation details
Does the local Hymba path include all original features and throughput optimizations?
Hymba: A Hybrid-head Architecture for Small Language Models
Show answer and example
Answer: No. It fuses causal attention and a depthwise convolutional state branch, without meta tokens, 1.5B scale, or fused kernels. A runnable branch does not establish paper throughput.
Illustrative example: Passing a small-model gated forward is a mechanism check, not a deployment speed measurement.
Sections in the Chinese source: 复现边界. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Method comparison
How does parallel Hymba differ from alternating attention and SSM layers?
Hymba: A Hybrid-head Architecture for Small Language Models
Show answer and example
Answer: Parallel branches both read the same layer input and fuse in one update. Alternating layers transform the first branch's result before the second computation. The graphs and fusion times differ at equal layer count.
Illustrative example: Two simultaneous readers share the original, unlike a second reader receiving only the first reader's summary.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Basics
Why combine explicit historical reads and recurrent summaries?
Hymba: A Hybrid-head Architecture for Small Language Models
Show answer and example
Answer: Attention directly retrieves positions, while the SSM compresses history into state. The same layer can use both without forcing either to carry every dependency.
Illustrative example: Explicit access can recover a name while recurrence carries grammar context.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why build sized virtual items before candidate-related retrieval?
TWIN V2: Scaling Ultra-Long User Behavior Sequence Modeling for Enhanced CTR Prediction at Kuaishou
Show answer and example
Answer: Offline clustering compresses repeated events. GSU retrieves relevant clusters and ESU performs size-aware interactions without naively duplicating large groups. Rare details may disappear.
Illustrative example: Represent years of similar shoe browsing with a count rather than recomputing attention on thousands of events.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Does public full-catalog ranking verify million-event serving?
TWIN V2: Scaling Ultra-Long User Behavior Sequence Modeling for Enhanced CTR Prediction at Kuaishou
Show answer and example
Answer: No. It tests candidate search and hierarchical compression without private lifelong logs or serving. Align memberships, sizes, and time masks.
Illustrative example: Small MovieLens retrieval is not production latency evidence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does cluster retrieval differ from SIM raw-behavior search?
TWIN V2: Scaling Ultra-Long User Behavior Sequence Modeling for Enhanced CTR Prediction at Kuaishou
Show answer and example
Answer: Cluster search filters prototypes before finer interaction; raw retrieval selects original behaviors. Compression changes coverage, accuracy, and cost, with possible rare-detail loss.
Illustrative example: A large cluster may hide a niche event that raw search can still find.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why compress lifelong history into sized virtual items?
TWIN V2: Scaling Ultra-Long User Behavior Sequence Modeling for Enhanced CTR Prediction at Kuaishou
Show answer and example
Answer: Per-event matching over millions is expensive. Offline hierarchy summarizes groups and tracks size, then online retrieval and ranking read relevant clusters.
Illustrative example: Many repeated events should differ from one isolated event.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Core mechanism
Why does a unified event stream make editor, terminal, and browser workflows auditable?
Show answer and example
Answer: Actions and sandbox returns enter one timeline. Policies continue from observed results and reviewers can trace changes and validation. Shared events do not erase tool-specific semantics or establish code safety and correctness. Isolation, permissions, and tests have distinct roles.
Illustrative example: Example: Editor events change a file, terminal events run tests, and browser events check a page. The stream shows whether tests preceded or followed edits.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Implementation details
Does a local browser interface mean real web interaction is connected?
Show answer and example
Answer: No. Local work emits editor events and runs fixed regressions in an isolated temp directory. Browser action semantics are retained without a live environment or full controller. Claim only observed events and outcomes, not real navigation from a type definition.
Illustrative example: Example: Recording browser_open does not open a page, fetch its DOM, or click an element. Those require real browser evidence.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Method comparison
Why are platform runtime and agent controller distinct contributions?
Show answer and example
Answer: The runtime supplies actions, observations, and isolation. A controller chooses plans and tool usage; different policies can share one platform. Running editor and terminal verifies execution paths, not full model reasoning or official SWE-bench resolution.
Illustrative example: Example: A script and LLM may use the same sandbox without equal issue understanding or repair ability.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Basics
Why is OpenHands a software-agent work platform for editing, execution, and browsing?
Show answer and example
Answer: Real software tasks require inspecting files, commands, pages, and continuing long workflows, not only generating code. OpenHands connects operations to feedback in a consistent environment. Platform support and a controller's ability are separate evidence; tools alone do not make policies reliable.
Illustrative example: Example: After editing a frontend, start the service, inspect the page, and return to code when it fails. Code generation alone misses that loop.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why must CDM's student see same-request candidates when distilling an MMR teacher?
Contextual Distillation Model for Diversified Recommendation
Show answer and example
Answer: MMR selection depends on other candidates and redundancy, so isolated item features cannot recover it. CDM learns selection propensity relative to context and combines it with the original recommendation score, transferring costly iterative redundancy reduction to a serving-time scorer. Student approximation need not reproduce the list exactly, and a small local run does not establish production diversity gains.
Illustrative example: A travel clip may be redundant among ten similar clips but distinctive in a cooking list.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Why update similarity penalties after selecting each item in the CDM teacher?
Contextual Distillation Model for Diversified Recommendation
Show answer and example
Answer: MMR penalizes maximum similarity to the current selected prefix. Every addition changes remaining marginal scores; one fixed ranking is not the same contextual decision. Student targets must correspond to the relevant prefix rather than conflating stages in a static item label.
Illustrative example: After selecting a camping tutorial, reduce the score of its near-duplicate. An empty-prefix score could put both next to each other.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
What does CDM distillation save compared with online MMR, and what error can it introduce?
Contextual Distillation Model for Diversified Recommendation
Show answer and example
Answer: MMR recomputes redundancy against the selected set at each step. The student approximates those decisions quickly, reducing iterative serving cost. It can drift on new candidate distributions or longer lists, so test precision and diversity as well as latency. Proxy gains do not prove exact imitation.
Illustrative example: One student score can replace repeated comparisons, but the student may underestimate redundancy in a near-duplicate set absent from training.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why does CDM view diversity as marginal value given the selected prefix?
Contextual Distillation Model for Diversified Recommendation
Show answer and example
Answer: Value depends on what the list already contains: the first science-fiction item can add variety while the fourth similar one is redundant. MMR combines relevance with redundancy penalties; CDM learns the contextual gain for low-latency serving. Diversity is not a permanent per-item bonus.
Illustrative example: A third movie by a director may add little if two are already selected, yet be valuable when no work by that director appears.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Core mechanism
Why might counterfactual watch-time scoring better capture preferences than raw watched seconds?
Counteracting Duration Bias in Video Recommendation via Counterfactual Watch Time
Show answer and example
Answer: Raw seconds combine attraction with the viewing opportunity allowed by duration. CWM estimates potential watch benefit under a common duration intervention. This depends on counterfactual assumptions, is not just dividing by duration, and local proxies do not establish that actual duration or exposure confounding was removed.
Illustrative example: Completing a 20-second clip and abandoning a long clip after 30 seconds may imply a different preference under equal viewing opportunity.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Implementation details
How should training and prediction duration differ for CWM's `do(D=d0)`?
Counteracting Duration Bias in Video Recommendation via Counterfactual Watch Time
Show answer and example
Answer: Fit the duration mechanism using observed D; for prediction comparisons fix d0 and output the defined potential quantity and bias term. Using each item's original duration is not a common intervention. Local movie proxies do not measure real short-video potential watch benefit.
Illustrative example: Set the same d0 for a 30-second clip and a five-minute clip; setting 30 and 300 reintroduces the original difference.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Method comparison
How do CWM and completion-rate ranking differ in their assumptions about duration?
Counteracting Duration Bias in Video Recommendation via Counterfactual Watch Time
Show answer and example
Answer: Completion rate is an observed ratio and can favor short content; it has no explicit counterfactual. CWM estimates a potential outcome under common duration to separate preferences and duration effects. Bias differs, and misspecification can prevent the causal objective from helping.
Illustrative example: A three-second ad is easy to complete, unlike a long tutorial. CWM instead asks which would offer more value at equal duration.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Basics
How does CWM's potential watch benefit under common duration differ from observed watched seconds?
Counteracting Duration Bias in Video Recommendation via Counterfactual Watch Time
Show answer and example
Answer: Observed seconds mix preference, duration constraints, and content-type relationships. A common duration intervention compares potential outcomes at equal opportunity, reducing inherent long-video advantage. It is unobserved and requires modeling and identification assumptions.
Illustrative example: Completing a 15-second clip versus watching two minutes of a longer clip cannot determine preference from seconds alone.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Core mechanism
Without a reference, how does SimPO avoid favoring short responses by sequence probability?
Show answer and example
Answer: Total log probability accumulates per token and changes reward scale with length. Average per token and require the preferred response to beat the rejected one by a margin, without an extra reference forward. This removes one length effect, not all quality bias.
Illustrative example: A short wrong answer can beat a longer correct one on summed log probability; averaging removes the direct sum-length penalty.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Implementation details
Which tokens enter SimPO's log-probability sum and length?
Show answer and example
Answer: Use the same valid-response mask for both; no padding or prompt tokens. The loss updates only the policy. A token objective does not reproduce the paper's judge/model matrix.
Illustrative example: Counting padding in the denominator distorts the short answer's average reward.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Method comparison
Is SimPO's training margin equivalent to reducing inference temperature?
Show answer and example
Answer: No. Margin sets desired average-reward separation and changes parameter gradients. Temperature rescales next-token logits at inference. They can coexist but shorter output at one temperature does not validate the margin.
Illustrative example: Training compares complete response average log probabilities; inference temperature doesn't read that preference pair.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Basics
Does reference-free SimPO mean there is no training feedback?
Show answer and example
Answer: No. Chosen/rejected pairs still supply preferences. It removes a resident reference model, using length-averaged policy log-probability as implicit reward with a fixed margin.
Illustrative example: You still need a better answer label; no frozen reference scores need be computed.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why does LEARN freeze LLM content encoding but train a causal PCH to learn preferences?
Show answer and example
Answer: A frozen content encoder retains open-world semantics at lower update cost, but semantic closeness is not co-consumption. PCH learns order and next-item prediction from historical content vectors, then projects them into recommendation space. Respect history cutoffs; freezing the LLM does not mean nothing is trained or that semantic similarity proves transfer.
Illustrative example: Illustration: Two films mention adventure, but the user's viewing sequence can reveal a more specific storytelling preference that frozen text semantics alone do not express.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
Is a frozen content cache enough to prevent future leakage in PCH next-item prediction?
Show answer and example
Answer: No. Enforce a causal mask and cutoff in PCH: the next item belongs only on the target side and must not enter pooling. Align encoder and projection versions. Frozen parameters prevent gradient updates, not input leakage.
Illustrative example: Illustration: Putting the test movie's frozen vector into history before predicting it still leaks the answer.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
Why isn't LEARN's large gain over frozen semantic mean necessarily a cold-start gain?
Show answer and example
Answer: The mean lacks trainable preference modeling, so PCH can improve ranking by learning sequences. The source also shows strong head concentration, so gains may come from popularity rather than new-item understanding. Cold-start subgroups and matched controls are needed; overall NDCG does not prove gains for all groups.
Illustrative example: Illustration: Recommending popular movies can substantially beat a semantic average without helping a new film reach its audience.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why freeze the content LLM while letting LEARN's PCH learn?
Show answer and example
Answer: The LLM provides stable content semantics; recommendation objectives still need to adapt behavioral relationships. PCH reads the content sequence, models causal preference dependencies, and projects into serving space. Freezing saves cost but limits content adjustment. Pretrained semantics are not collaborative preferences and need not improve all cold-start cases.
Illustrative example: Illustration: Films of the same genre may have different audiences. PCH can learn the difference from viewing order, whereas averaging frozen vectors does not train preference structure.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Core mechanism
Why can a specialized ACI work better than an unrestricted shell?
Show answer and example
Answer: Broad actions and verbose output increase selection and localization load. An ACI makes search, viewing, precise edits, and tests easier to act on and interpret. This improves the repository interaction contract, not hidden patch access. Real tests still verify completion.
Illustrative example: Example: Showing nearby lines and an exact replacement makes editing easier than printing a whole file, but regression tests must validate the patch.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Implementation details
What do the initial failure and fixed unittest prove in local swebench-local?
Show answer and example
Answer: A real initial failure shows a problem exists. The exit code after editing verifies the covered regression. Local tools only read files, write solution.py, and run fixed tests. This is executable micro-benchmark evidence, not official SWE-bench Lite or broader coverage.
Illustrative example: Example: Fixing arithmetic until one suite passes verifies the edit-and-run path, not official GitHub issue resolved rate.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Method comparison
Why compare ACI with raw shell using the same model and tasks?
Show answer and example
Answer: Otherwise stronger models or easier tasks confound interface effects. ACI changes action and observation boundaries, so control other conditions and assess actual patches and tests. One good edit or valid command syntax cannot establish broad repair ability.
Illustrative example: Example: Give the same model the same bug with a narrow editor versus a raw shell to isolate differences in localization and editing.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Basics
Why does a GitHub issue agent need a convenient repository viewing and editing interface?
Show answer and example
Answer: The model must locate unknown code, read needed context, make small edits, and verify. Raw terminal output can be long or fragile. SWE-agent's interface better fits that workflow without replacing issue understanding or tests. A successful command does not mean resolution.
Illustrative example: Example: Open matching files and relevant lines rather than printing the repository, then run the target tests after edits.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why does BAHE still train higher-layer aggregation when shallow encodings of atomic behavior are cached?
Breaking the Length Barrier: LLM-Enhanced CTR Prediction in Long Textual User Behaviors
Show answer and example
Answer: Shallow layers encode reusable text semantics for individual actions; higher layers need the current history and candidate and cannot be frozen into a universal answer. This moves repeated text computation off training and request paths while retaining interactions. Stale or excessively compressed cache entries can harm quality despite better efficiency.
Illustrative example: Cache a movie title once, but aggregate it differently when watched alongside science fiction or family movies and when scoring different candidates.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
How does the local two-layer BERT-tiny BAHE split caching and training?
Breaking the Length Barrier: LLM-Enhanced CTR Prediction in Long Textual User Behaviors
Show answer and example
Answer: The first layer generates and caches atomic MovieLens title vectors. The copied upper layer processes CLS, 12 history entries, and the candidate. The control runs frozen lower and trainable upper layers per sample with matched data and budget. Bind the cache to the shallow version; a two-layer timing ratio does not predict production-scale gains.
Illustrative example: If the upper layer expects a different shallow encoder version from the cache producer, vector semantics can mismatch. Record the vector and upper-layer versions together.
Sections in the Chinese source: 核心公式, 本地复现. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
Where does BAHE save computation compared with rerunning the full text model per sample, and what can it lose?
Breaking the Length Barrier: LLM-Enhanced CTR Prediction in Long Textual User Behaviors
Show answer and example
Answer: Repeated shallow encoding is avoided, and upper layers process actions rather than every word token. Compression removes detail and changes capacity, so measure time and quality together. Local time fell but AUC also fell, supporting the efficiency mechanism rather than a quality-preserving substitution.
Illustrative example: Ten titles may contain over a hundred words; ten cached behavior vectors are quicker to process, but losing a critical phrase can damage ranking.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why is BAHE's atomic behavior cache different from caching the final user representation?
Breaking the Length Barrier: LLM-Enhanced CTR Prediction in Long Textual User Behaviors
Show answer and example
Answer: An atomic cache holds shallow single-item content encodings reusable across histories. The final user representation also depends on order, the current candidate, and higher-layer interactions. Asynchronous shallow updates remove repeated text work; the upper layer still learns personalized combinations. Finer caching increases reuse but requires version consistency and care over compression losses.
Illustrative example: Two users can read the same cached title vector in different history orders without sharing final interest scores.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Core mechanism
How do Data Mixing Laws search untrained mixtures, and what limits them?
Data Mixing Laws: Optimizing Data Mixtures by Predicting Language Modeling Performance
Show answer and example
Answer: Small matched runs observe domain losses, a fitted relation predicts mixture/scale performance, and a weighted objective searches it. This avoids exhaustive large runs but depends on pilot coverage and assumptions. Keep final test isolated.
Illustrative example: Use pilots at 20%, 50%, and 80% code to select a few intermediate candidates; predictions are not measured large-model results.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Implementation details
How keep test outcomes out of mixture selection?
Data Mixing Laws: Optimizing Data Mixtures by Predicting Language Modeling Performance
Show answer and example
Answer: Fit and choose domain weights/candidates under prespecified train/validation rules. Test once at the end. Mixtures are nonnegative and sum to one. A unigram proxy is not RedPajama scaling.
Illustrative example: Do not retune w_i after seeing test domain losses.
Sections in the Chinese source: 核心公式, 复现边界. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Method comparison
How does fitted mixture search differ from measuring every mix?
Data Mixing Laws: Optimizing Data Mixtures by Predicting Language Modeling Performance
Show answer and example
Answer: Fitted predictions save compute but can have approximation and extrapolation error. Direct runs give measured outcomes at higher cost. Independently validate a predicted optimum.
Illustrative example: A predicted best 70/30 mix has not necessarily trained at target scale.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Basics
Why train small pilots across mixtures?
Data Mixing Laws: Optimizing Data Mixtures by Predicting Language Modeling Performance
Show answer and example
Answer: Pilots observe each domain and support fitting mixture and scale effects. This reduces exhaustive runs but depends on coverage and modeling assumptions.
Illustrative example: Ten pilot mixes can shortlist others, not turn untrained points into measurements.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why give LSVCR personalized LLM preferences during training if no generative LLM runs at serving time?
A Large Language Model Enhanced Sequential Recommender for Joint Video and Comment Recommendation
Show answer and example
Answer: Video and comment semantics differ and may exceed what ID sequences capture. Cross-fusion and alignment transfer teacher knowledge into a deployable dual-sequence model. Tasks can interfere: local video and comment metrics move in different directions, so neither alone establishes overall success.
Illustrative example: Illustration: A teacher can explain sarcasm in a comment during training; the serving student must judge from available sequences without secretly consulting it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
Why sample an ordered subset for LSVCR position encoding instead of shuffling positions?
A Large Language Model Enhanced Sequential Recommender for Joint Video and Comment Recommendation
Show answer and example
Answer: A short alignment sequence should cover the span of the full position table without changing chronology. An ordered subset samples uneven gaps; shuffling changes temporal relations. Cross-sequence fusion must also exclude target video and comment events not yet observed.
Illustrative example: Illustration: Assign positions 3, 20, and 80 from a 1–100 table. Assigning 80, 3, and 20 instead changes which event appears newest.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
What relationships do LSVCR's SSC and VCC constrain?
A Large Language Model Enhanced Sequential Recommender for Joint Video and Comment Recommendation
Show answer and example
Answer: SSC aligns the sequential recommender's user preferences with the LLM; VCC links video or item preferences with comment preferences. They operate across models and across objects, respectively, and can conflict. The local comment metric improved while the item metric declined; report both.
Illustrative example: Illustration: Learning the teacher's photography preference is different from deciding whether a user likes photography products and photography comments equally.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why encode video and comment consumption separately in LSVCR instead of treating comments as video metadata?
A Large Language Model Enhanced Sequential Recommender for Joint Video and Comment Recommendation
Show answer and example
Answer: Comments have their own reading behavior and semantics. Dual sequences preserve the distinction before cross-fusion, and the training-time LLM adds personalized knowledge. Co-consumption can correlate or conflict, so better comment ranking does not prove better video ranking.
Illustrative example: Illustration: Someone may dislike a video but enjoy its discussion. Adding comment text only to a video description can lose this independent preference.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Core mechanism
What do SFT and the odds-ratio term each constrain in ORPO?
Show answer and example
Answer: SFT raises chosen likelihood; the odds term widens separation from rejection. Both update one policy without reference in the objective. Odds are boundary-sensitive. Reporting reference KL as a diagnostic does not add it to training.
Illustrative example: Raising only chosen likelihood can also raise similar bad answers; odds explicitly separate them.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Implementation details
Why handle probabilities near zero or one in ORPO's log-odds margin?
Show answer and example
Answer: Log odds are log(p)−log(1−p), then contrasted across replies. Boundaries cause log zero or huge ratios. The local kernel adds 1e−12 to both terms and uses logaddexp, not probability subtraction. Candidate probabilities are not token-level response probabilities.
Illustrative example: 0.9→0.99 is +0.09 probability but odds go 9→99.
Sections in the Chinese source: 核心公式, 复现边界. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Method comparison
How does ORPO's single stage differ from SFT followed by separate preference optimization?
Show answer and example
Answer: One policy update combines chosen NLL and preference odds without a resident frozen reference for another stage. Two-stage training first produces an SFT policy then changes objective/reference. One stage is not universally better.
Illustrative example: One preference batch supplies NLL and odds gradients instead of finishing chosen-only SFT first.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Basics
What is the difference between ORPO odds and probability?
Show answer and example
Answer: Probability p is an event's share of all outcomes. Odds p/(1−p) compare occurrence to nonoccurrence and can exceed one. ORPO compares preferred and rejected odds, not percentages.
Illustrative example: p=1/2 gives odds 1; p=3/4 gives odds 3, not 300% probability.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why train NoteLLM's compression state on co-read contrastive learning and content generation?
NoteLLM: A Retrievable Large Language Model for Note Recommendation
Show answer and example
Answer: Co-read pairs capture collaborative relations but contrastive similarity alone can lose fine semantics. Generating tags and categories from the same bottleneck forces it to preserve recoverable content. The two objectives shape a retrieval vector that does not merely memorize co-occurrence.
Illustrative example: Illustration: Tent and cookware notes can be close for campers yet retain enough detail to generate distinct tags.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
How verify NoteLLM's contrastive vector and CSFT initial state use the same compression position?
NoteLLM: A Retrievable Large Language Model for Note Recommendation
Show answer and example
Answer: Read the same special-token hidden state for both, not a last position for one and mean for the other. Align batch pairs and causal generation masks and exclude padding. Frozen versus joint training tests representations, not a full DIN ranking comparison.
Illustrative example: Illustration: Title-mean contrastive features and an independent CLS for generation can both train without constraining the same bottleneck.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
What relation does GCL co-reading add beyond text-similarity positives?
NoteLLM: A Retrievable Large Language Model for Note Recommendation
Show answer and example
Answer: Text similarity is content-driven; co-reading links consumption across superficially different notes but can reflect exposure and popularity. CSFT constrains content retention. Complementarity is not absence of bias, and simplified labels do not reproduce real hashtag generation.
Illustrative example: Illustration: Tent reviews and camp recipes may be co-read despite different text, whereas two near-identical tent reviews are textual matches.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why is NoteLLM's compression token both a retrieval and generation interface?
NoteLLM: A Retrievable Large Language Model for Note Recommendation
Show answer and example
Answer: Contrastive retrieval and label generation use the same compressed state, requiring collaborative and semantic evidence. If generation bypasses it to read raw text, the constraint changes. Compression does not preserve every note detail.
Illustrative example: Illustration: A token must bring co-read notes together and still generate 'camping gear,' not just encode shared popularity.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why remove softmax but normalize after aggregation and apply a U gate?
Show answer and example
Answer: Softmax fixes weight mass and can suppress behavior-frequency intensity. Pointwise SiLU aggregation preserves variation; post-normalization stabilizes and the token-conditioned U gate modulates output. Causal window interactions are not linear complexity.
Illustrative example: One versus ten related events can differ before the gate controls how much information passes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
What does M-FALCON share, and why not cache every ranking score?
Show answer and example
Answer: Section 3.4/Appendix H isolate candidate microbatches and share per-layer candidate-independent history KV. Each candidate still projects and interacts separately. Versions and history constrain reuse; local HSTU omits this serving optimization.
Illustrative example: A thousand items can share history while cameras and shoes retain different scores.
Sections in the Chinese source: 背景与主要改动, 本地复现. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
How does all-position HSTU training preserve causality?
Show answer and example
Answer: Each valid state reads its prefix and predicts the next item; mask padding and invalid targets. One encoding supplies multiple losses. Candidate action interleaving is a separate contract.
Illustrative example: A predicts B and A/B predicts C without A seeing B or C.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Do HSTU history tokens interact, unlike a static history vector?
Show answer and example
Answer: Yes. Each position updates by reading visible earlier tokens, producing contextual history that a target later reads. Causality restricts direction, not history interaction; outside-window events remain absent.
Illustrative example: A lens event can incorporate earlier camera context before a tripod candidate reads it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
Why not interpret local matched-budget SASRec losses as industrial superiority?
Show answer and example
Answer: Matched width, layers, loss, and steps isolate a local core. MovieLens lacks heterogeneous events, timing, long histories, and scale. Preserve local negatives without dismissing paper production evidence or importing it.
Illustrative example: A short-film-sequence loss neither predicts 100,000-event failure nor becomes a local win from online results.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why do ranking and retrieval predict different objects in HSTU?
Show answer and example
Answer: Retrieval predicts a relevant next item. Ranking conditions on the candidate to predict actions, interleaving item/action tokens. Full-catalog next-item metrics do not validate every action task.
Illustrative example: Guess a film versus predict clicks or long watch for an already given film.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
Why exclude the current response from RLOO's same-prompt baseline?
Show answer and example
Answer: Peers provide an independent reference, not the current action's random reward. Treating a response as one action avoids a value network but still requires enough samples, consistent rewards and any specified reference/evaluator.
Illustrative example: For reply one of four, average the other three so the baseline doesn't move with its own reward.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
Can RLOO use the same formula at K=1?
Show answer and example
Answer: No. K−1 is zero and there are no peers. Require sufficient group size and KL-shaped returns as appropriate. Local K=4 candidates aren't 7B generation and real RM.
Illustrative example: Zero-filling an empty baseline changes the estimator.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
How does leave-one-out differ from a mean including the current sample?
Show answer and example
Answer: LOO excludes the sample's own reward from its baseline. Including it changes dependency and scaling. Both are critic-free, but group size matters.
Illustrative example: With two replies, the first's LOO baseline is the second, not both averaged.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
Why does RLOO treat a full response as one action?
Show answer and example
Answer: Terminal RLHF rewards suit response-level REINFORCE and avoid prefix values. Other same-prompt samples form a variance-reducing baseline; multiple samples are still needed.
Illustrative example: A summary's terminal score weights its complete logp relative to peers.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
How does GRPO obtain advantages without critic, and what are old and reference for?
Show answer and example
Answer: Center and scale group rewards for relative signal. Old gives sampling probabilities for ratio/clipping; reference anchors KL. Equal rewards provide no contrast. Candidate updates don't reproduce full math training.
Illustrative example: A reference change alters regularization, not the historical rollout source.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
What should GRPO do with equal group rewards?
Show answer and example
Answer: Centered advantages are zero and SD needs protection; never invent differences using evaluator gold. Local six-action candidates aren't DeepSeekMath generation.
Illustrative example: Don't rank four incorrect attempts by gold proximity unless the reward protocol actually says so.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
Why are GRPO old and reference policies different roles?
Show answer and example
Answer: Old supplies rollout and proximal ratios, reference anchors long-term KL. Old can refresh each batch independently of the reference; mixing couples update limits and regularization.
Illustrative example: Fresh sampling updates old policy, not necessarily initial reference.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
Why can GRPO omit a policy-sized value model?
Show answer and example
Answer: Same-prompt group mean and SD estimate relative advantages instead of a learned prefix value. Save critic resources but require repeated rollouts and differing feedback.
Illustrative example: Better attempts receive positive advantage relative to peers.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Core mechanism
How does KTO update from single desirable feedback without same-prompt pairs?
Show answer and example
Answer: Use response policy/reference log ratio against expected KL as a dynamic baseline and define separate desirable/undesirable utilities. Independent examples can be reweighted without fictitious comparisons. Reference and imbalance still matter; a label is not absolute reward.
Illustrative example: Upvote and downvote on separate inquiries enter separate branches, not a fabricated preference pair.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Implementation details
What is KTO's reference point, and what does the local policy establish?
Show answer and example
Answer: It is expected policy-to-reference KL, not one answer's log ratio. Separate aggregate baseline and sample signal. The local six-candidate linear policy checks this relation, not free generation.
Illustrative example: Compare a high log ratio to the moving baseline; don't treat the example as the entire KL expectation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Method comparison
Does KTO differ from DPO only by unpairing the data?
Show answer and example
Answer: No. KTO defines gains/losses around a KL reference point; DPO compares same-prompt response margins. Standalone feedback cannot be arbitrarily paired, and class weights alter tradeoffs.
Illustrative example: One user's upvote and another's downvote on different queries do not order their answers to one prompt.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Basics
Are KTO desirable and undesirable weights feedback probabilities?
Show answer and example
Answer: No. Labels describe single accepted/rejected outputs; λ weights control class utility contributions. Class frequency and explicit weight are separate, neither a predicted satisfaction probability.
Illustrative example: Rebalance many upvotes and few downvotes without inventing matched same-question pairs.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why verify a candidate tree rather than concatenate each Medusa head's top token?
Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads
Show answer and example
Answer: Heads predict different offsets from the same state and may not form a valid autoregressive chain. A tree exposes combinations for backbone validation. Independent losses do not prove correctness or sampling guarantees, and local work has no GPU tree kernel.
Illustrative example: Separately chosen nouns and verbs need not form a coherent phrase.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
How align Medusa's k-th head targets?
Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads
Show answer and example
Answer: Predict x_{t+k} from h_t, matching slices and valid endpoints. One common next-token target would train duplicate offsets.
Illustrative example: Head two at h_10 predicts x_12, not x_11.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Method comparison
Why don't fewer backbone calls imply the same percentage latency reduction?
Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads
Show answer and example
Answer: Extra heads, tree construction and validation, batch shape, kernels, and acceptance all matter. Mechanism call counts are not measured GPU speed.
Illustrative example: Halving calls may double the work per verification.
Sections in the Chinese source: 本地复现, 复现边界. The example illustrates the mechanism.
Basics
Why add multiple future-token heads to one hidden state?
Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads
Show answer and example
Answer: Different offsets provide multiple proposals at once. Tree verification can reduce sequential calls without removing backbone verification.
Illustrative example: Propose t+1 and t+2 from h_t before checking accepted combinations.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Core mechanism
Where do Self-Rewarding LM preferences come from, and why can errors compound?
Show answer and example
Answer: Each round ranks its own generated candidates and trains the next round with DPO on those judgments. They are not independent human labels; persistent judge biases can become stronger. Keep external held-out evaluation.
Illustrative example: A judge rewarding verbosity may repeatedly choose long arithmetic errors; an external check can break the loop.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Implementation details
What provenance must Self-Rewarding iterations retain?
Show answer and example
Answer: Record generating policy round, judge round and pair-selection rule. Don't backfill old data with post-update scores. Use isolated evaluation. Local tests cover state and objective, not paper judge quality.
Illustrative example: After training a new model, don't rescore old candidates and present those scores as original supervision.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Method comparison
How does Self-Rewarding differ from a fixed external reward model?
Show answer and example
Answer: A fixed judge doesn't coevolve; self-reward hopes both generation and judgment improve but can amplify shared errors. Compare at equal budget with independent evaluation, not rising self-scores.
Illustrative example: A model can learn to please its own scoring rule without becoming more useful to users.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Basics
Does a Self-Rewarding judge tie reveal the preferred answer?
Show answer and example
Answer: No. A tie gives no strict preference; display order cannot create one. Candidate count and distinguishable judging are separate. Even unequal scores are the current judge's judgment, not new human truth.
Illustrative example: If two candidates tie, don't label the first shown the winner.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Core mechanism
How does SPIN's previous-policy negative differ from self-judged preferences?
Show answer and example
Answer: Human demonstrations are positives; prior-policy samples are negatives. Train relative to the prior policy, then refresh it. No self-judge scoring is needed. Human supervision remains and negatives aren't necessarily false.
Illustrative example: Use fixed expert examples and regenerate opponents next round instead of replaying all old negatives.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Implementation details
Why freeze SPIN's opponent during a round and refresh it afterward?
Show answer and example
Answer: The previous policy supplies both samples and the reference. Changing it mid-round changes the objective. Refresh after the round to align generation, scoring and updates. No full-scale local text experiment.
Illustrative example: Using old reference for half the batches and a new one for the rest does not define one consistent round.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Method comparison
How does SPIN's refreshed negatives differ from repeated SFT on fixed examples?
Show answer and example
Answer: SFT raises fixed demonstration likelihood. SPIN resamples a comparison distribution from the previous policy on the same prompts and distinguishes demonstrations from it. Both negatives and relative reference change, not just training rounds.
Illustrative example: After fixing earlier mistakes, the next opponent may make different ones. Fixed SFT won't refresh those negatives.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Basics
Where do SPIN positives come from without new human preference data?
Show answer and example
Answer: Existing human SFT demonstrations stay positive. The previous policy generates negatives on the same prompts. Self-play changes negatives and reference; it still uses demonstrations and generated replies are not necessarily factually wrong.
Illustrative example: A standard answer already exists; generate a comparison without asking a human to rank it again.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Core mechanism
Why isn't a continuation-based step label a direct correctness judgment?
Show answer and example
Answer: It estimates success under a particular policy and sampling budget. Correct prefixes can get low estimates and wrong ones can be repaired. Separate reachability from correctness; use gold only for annotation or independent scoring.
Illustrative example: Low success after a correct second step may reflect frequent final arithmetic mistakes, not a wrong second step.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Implementation details
Can Math-Shepherd continuations see gold when constructing labels?
Show answer and example
Answer: No. Gold is only for terminal checking; actually sample from the specified prefix rather than constructing a path from known answers. No full local LLM data/PRM training.
Illustrative example: The grader can know the result without including it in the continuation prompt.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Method comparison
How does continuation success differ from correctness of a step?
Show answer and example
Answer: Later policy can repair an error or fail after correct reasoning. Estimated reachability combines prefix and continuation policy, not a logical proof.
Illustrative example: A wrong step followed by frequent corrections should not be imitated simply because continuations succeed.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Basics
How does Math-Shepherd get step supervision without human step labels?
Show answer and example
Answer: Continue repeatedly from a prefix and estimate reachable success from final answer checks, then train a process verifier. This is not direct human judgment of each line.
Illustrative example: A prefix with mostly correct continuations receives a higher success estimate.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Core mechanism
What does Mamba's selectivity change inside the SSM?
Mamba: Linear-Time Sequence Modeling with Selective State Spaces
Show answer and example
Answer: Token-dependent step sizes and read/write vectors control retention, write direction, and readout within recurrence. This is not output scaling of a fixed SSM. Linear sequence cost still does not guarantee exact arbitrary-token retrieval from finite state.
Illustrative example: Preserve state on filler but write differently for a key number; retrieval still depends on what remains.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Implementation details
Does a Python or PyTorch selective scan reproduce Mamba kernel throughput?
Mamba: Linear-Time Sequence Modeling with Selective State Spaces
Show answer and example
Answer: No. A loop can check input-dependent parameters and recurrence, but not a fused parallel CUDA scan. The local run also omits 2.8B pretraining. Validate correctness, language quality, and throughput separately.
Illustrative example: Seven steps agreeing with a reference check mechanics, not tokens per second.
Sections in the Chinese source: 本地复现, 复现边界. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Method comparison
How does Mamba memory differ from reweighting historical KV?
Mamba: Linear-Time Sequence Modeling with Selective State Spaces
Show answer and example
Answer: Mamba accumulates history in state and uses input-dependent readout. Attention assigns weights directly to retained positions. Linear complexity does not imply lossless per-token history.
Illustrative example: A summary can retain a fact without separately addressable slots for every old word.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Basics
Why does Mamba selectivity depend on the current token?
Mamba: Linear-Time Sequence Modeling with Selective State Spaces
Show answer and example
Answer: Step size and read/write vectors depend on input, so retention, updates, and readout vary by content while still using recurrent state.
Illustrative example: A separator and answer token can trigger different updates.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Core mechanism
Why can exact short answers still test multi-tool reasoning rather than memorization?
Show answer and example
Answer: Short output is the scoring interface; finding it can require browsing, multimodal understanding, calculation, and cross-source checks. Normalized matching makes endpoints clear, not evidence quality or failure location. Distinguish accuracy from executed traces and never put reference answers into agent prompts.
Illustrative example: One final year may require identifying a person from a picture, researching an event, and verifying dates.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Implementation details
May GAIA normalization be relaxed arbitrarily after inspecting gold?
Show answer and example
Answer: No. Fix allowed normalization and task sets, and expose gold only to the scorer. Local interface or protocol checks lack real web and multimodal runs and cannot report benchmark scores.
Illustrative example: Changing rules because a wrong answer contains a correct substring changes the evaluation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Method comparison
How does normalized GAIA exact match differ from an LLM judge assigning semantic scores?
Show answer and example
Answer: Exact match checks equality after fixed normalization. A semantic judge decides equivalent meaning and has different tolerance and biases. Changing the evaluator cannot silently keep the same GAIA score. Predefine normalization instead of rewriting predictions after seeing gold.
Illustrative example: One normalizer handles permitted surface variation. Asking a model to accept a vague paraphrase changes the grader.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Evaluation and reliability: how do we know it worked?
Basics
Does GAIA expect a long reasoning transcript or a checkable short final answer?
Show answer and example
Answer: Score the final answer by exact comparison after prescribed normalization. Solving may still require browsing, images, and tools. Reasoning and submission format are distinct, so many plausible steps do not guarantee a match, and format failure cannot be hidden as reasoning success.
Illustrative example: If a question asks for a person, submit the checkable name. Search notes without a clear final name are not automatically correct.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Core mechanism
Why does BEQUE apply PRO to multiple self-sampled rewrites after SFT?
Large Language Model based Long-tail Query Rewriting in Taobao Search
Show answer and example
Answer: SFT learns demonstration rewrites, not necessarily their retrieval effects. BEQUE samples several candidates for each query, builds a preference ordering from offline search feedback, and uses length-normalized sequence scores to learn relative quality. The feedback combines objectives such as relevance and incremental recall; a composite win can hide a component regression.
Illustrative example: Two fluent rewrites may differ because one retrieves only existing popular items and the other adds relevant long-tail items. Search feedback detects that difference; also check incremental coverage.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Implementation details
Why must BEQUE's local PRO ordering exclude human SFT targets from its candidates?
Large Language Model based Long-tail Query Rewriting in Taobao Search
Show answer and example
Answer: Alignment evaluates actual model-generated rewrites. Inserting a gold target introduces an answer the generator might not have produced, changing the experiment. Use model candidates and a label-free fallback, rank by catalog feedback, and isolate training and test preference sets.
Illustrative example: If all four beams miss the right category, report that feedback; injecting the correct rewrite and selecting it disguises a generation failure.
Sections in the Chinese source: 核心公式, 本地复现. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Method comparison
How does BEQUE's PRO supervision differ from continuing SFT only on the best rewrite?
Large Language Model based Long-tail Query Rewriting in Taobao Search
Show answer and example
Answer: Winner-only SFT discards relations among other candidates for the same query. PRO contrasts ranked candidates successively, teaching probability ordering across quality levels while retaining SFT for stability. Length normalization and temperature affect the comparison, and the feedback design still limits what a composite improvement means.
Illustrative example: With four ordered rewrites, winner-only SFT imitates the first; PRO additionally learns that the second beats the third and the third beats the fourth.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Basics
Why not evaluate BEQUE rewrites only by their textual similarity to human rewrites?
Large Language Model based Long-tail Query Rewriting in Taobao Search
Show answer and example
Answer: Similar wording does not guarantee relevance, coverage, or hits in the retrieved item set. BEQUE scores self-sampled rewrites using a real search system and trains the generator on the induced preferences. Improvements in the chosen composite feedback do not guarantee that all metrics improve.
Illustrative example: A polished rewrite of a damp-proof camping query into outdoor lifestyle can lose sleeping-mat results. Inspect the retrieval, not the wording alone.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Core mechanism
Why fit a finite IPO gap rather than always separating chosen and rejected farther?
Show answer and example
Answer: Logistic losses can keep enlarging a separable margin, risking overfit. IPO squares error to a finite reference-relative log-ratio target. Overshooting reverses the gradient. This changes loss shape, not just learning rate.
Illustrative example: Once separation is sufficient, continued suppression of rejection need not help; excess separation is corrected.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Implementation details
Why use a cross policy/reference probability ratio in IPO?
Show answer and example
Answer: It measures how much the preferred-over-rejected relationship changed relative to reference, not just policy probability difference. Use a fixed positive τ and target 1/(2τ), aggregate response tokens correctly and retain reference terms.
Illustrative example: If reference already favors a wording, the cross-ratio distinguishes inherited preference from added change.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Method comparison
Does fitting IPO's gap require the chosen response's absolute probability to increase?
Show answer and example
Answer: No. The gap is chosen-minus-rejected policy/reference log ratios. Raising chosen, lowering rejected or both can reach the target. Chosen probability alone omits the relative comparison and global normalization.
Illustrative example: Chosen can stay fixed while rejected falls and the gap grows.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Basics
Does increasing τ increase or decrease IPO's target gap?
Show answer and example
Answer: The finite target is 1/(2τ), so increasing positive τ decreases the gap. It is not a label-strength score; training outcomes also depend on data, policy and optimization.
Illustrative example: Changing τ from 1 to 2 changes 1/2 to 1/4; this is formula arithmetic.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
Must the ReMax greedy baseline be correct?
Show answer and example
Answer: No. It is a variance-reducing comparator for the same prompt. Relative returns can be positive or negative independent of absolute correctness; baseline isn't gold. Both paths still cost generation and scoring.
Illustrative example: A wrong greedy and correct sample yield positive signal; two wrong replies do not make greedy the standard answer.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
Should ReMax differentiate through greedy baseline reward?
Show answer and example
Answer: No. Treat it as fixed return, not a gradient through discrete decoding or scoring. Use the same prompt/protocol for both. Local greedy candidates do not establish memory or latency.
Illustrative example: Do not invent an argmax reward gradient while updating the sampled policy.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
How do ReMax and RLOO baseline sources differ?
Show answer and example
Answer: ReMax additionally executes greedy decoding. RLOO uses other randomized responses. Both save the critic but spend generation differently and have different baseline errors.
Illustrative example: Compare one greedy rollout's cost to multiple peer samples.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
Why use same-prompt greedy reward as ReMax baseline?
Show answer and example
Answer: It gives a prompt-specific reference without training value. Reinforce samples above greedy and suppress below it; greedy need not be correct.
Illustrative example: A sampled explanation better than the usual first choice gives positive credit.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why actively manage page-in and page-out when the external archive is large?
Show answer and example
Answer: Archive capacity is not visible context. Relevant material must enter the working window to affect an answer. Paging links storage to the current task, and heartbeats or interrupts schedule further evidence gathering and user interaction. A virtual-memory interface does not let a model see unlimited history at once.
Illustrative example: Example: An old address remains in the archive. Retrieve and page it in before answering a shipping question.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
Does deterministic key-based page-in establish semantic archive retrieval?
Show answer and example
Answer: No. Local tests cover hierarchical state, paging, and interrupt counts, using deterministic keys instead of embeddings or LLM function calls. Proxy cost is not token count or latency. Paging works, but real autonomous evidence retrieval from long documents is unverified.
Illustrative example: Example: Getting an entry by order123 is easier than understanding and retrieving the user's vaguely mentioned last order.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
How does a heartbeat differ from a normal user-facing response in control flow?
Show answer and example
Answer: Heartbeat or interrupt continues internal memory and tool work rather than treating each operation as a final response. This is a scheduling interface, not a guarantee of correct invocation. Unconditional loops can waste budget, so use it only for relevant pending work.
Illustrative example: Example: After paging old content out, continue internal retrieval and answer once relevant evidence is loaded instead of returning a partial status as the answer.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Why does MemGPT compare long documents and conversations to limited-memory management?
Show answer and example
Answer: One context window is finite while user material can be much larger. MemGPT lets agents schedule information across visible working memory and external storage to continue long tasks. Archive capacity does not imply current knowledge; finding, paging, and storing information affect answers and cost.
Illustrative example: Example: To answer about a six-month-old contract, locate and page in the relevant record. Having the file on disk is not enough.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Core mechanism
Why condition generation on attributes instead of aggregating all rewards?
Show answer and example
Answer: A single scalar makes preference mixtures hard to change at inference. Predict attributes, then train conditional SFT so users can specify combinations. Labels must be reliable, and control does not guarantee precise independent satisfaction.
Illustrative example: Ask for detailed but restrained or concise and friendly without retraining a policy for each combination.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Implementation details
Are local SteerLM attribute labels equivalent to the authors' predictor?
Show answer and example
Answer: No. Deterministic reward axes on public candidates supply local labels to test the interface. No large attribute predictor or free-text conditioned generator is trained. Preserve attribute order and value domains.
Illustrative example: Swapping quality and length columns may run while reversing user controls.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Method comparison
How does SteerLM differ from asking for brevity in an inference prompt?
Show answer and example
Answer: Attribute-conditioned SFT systematically associates labeled values with response patterns. A one-off natural-language instruction needn't have that training signal. They can coexist, but adding attribute text alone is not conditioned training.
Illustrative example: Training examples across concision levels give an inference level learned meaning beyond wording.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Basics
Are SteerLM's attribute predictor and conditioned generator the same role?
Show answer and example
Answer: No. The predictor annotates existing answers with helpfulness, quality and other dimensions. The generator learns to produce answers under target attributes supplied at inference; it does not need a future answer scored first.
Illustrative example: Annotate training replies as concise/helpful, then request new replies conditioned on those attributes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Core mechanism
Why can UCT's exploration term give a low-scoring action another try?
Show answer and example
Answer: Mean value reflects observed returns. The exploration bonus grows as the parent is visited more and the child remains rare, giving uncertain branches a chance. Execution feedback updates values and failure reflection guides later expansion. Always taking the current best estimate can lock in a wrong branch, and model self-scores cannot replace environmental feedback.
Illustrative example: Example: Action A was mediocre across ten tries; B failed on its only try. UCT may try B again to reduce uncertainty rather than eliminate it permanently.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Implementation details
Is local search over leaves that are complete plans equivalent to the paper's LM-value search?
Show answer and example
Answer: No. The local path retains MCTS statistics, environmental reward, reflection, and backtracking but uses complete plans as leaves. It does not call an LM value function, the HumanEval executor, or WebShop. A perfect deterministic score verifies the search path, not code-generation or web-task success.
Illustrative example: Example: Choosing a valid route from four premade plans differs from generating actions and evaluating intermediate nodes on a real website.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Method comparison
How does LATS's MCTS allocate budget differently from retaining several thoughts per layer?
Show answer and example
Answer: MCTS repeatedly selects branches from the current tree, runs rollouts, and backs up returns and visit counts, directing future budget to paths worth revisiting. Layer-wise search usually compares and prunes a frontier at one depth. Both depend on candidate and evaluation quality; different organization and feedback use do not guarantee one wins on a given task type.
Illustrative example: Example: A depth-two route nearly succeeded. MCTS can deepen it later, while a fixed-width search may retain only the best currently rated candidates at that layer.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Basics
Why is one plausible reasoning chain insufficient for the tool tasks LATS addresses?
Show answer and example
Answer: Tools or environments may return unexpected results, and a single route may not recover after failure. LATS puts reasoning and actions in an explorable search tree, tests candidates through execution feedback, and uses failure reflection to try alternatives. It links proposals to environment results, though search budget and judgment errors still matter.
Illustrative example: Example: The first search result lacks a price. Return to an earlier search node and revise the keywords instead of inventing an answer from the wrong page.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Core mechanism
Why swap candidate order for RLAIF judging, and can raw scores be averaged?
Show answer and example
Answer: Judges may favor earlier positions. Evaluate both orders and combine only after mapping to the same candidate identity. Raw positional averaging can cancel a true preference. It controls position, not factual, style or value biases.
Illustrative example: Convert both trials to A's win score before averaging.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Implementation details
How does local RLAIF keep identities straight when swapping order?
Show answer and example
Answer: Map each judgment back to original candidate IDs before combining labels. Fixed labels verify mapping and update, not a full judge or large RL.
Illustrative example: If left wins on pass two and B is now left, credit B rather than A.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Method comparison
How does RLAIF training differ from an AI choosing an answer at inference?
Show answer and example
Answer: Training converts judgments to data and updates parameters. Inference selection changes one response without learning. Similar judges may be used, but costs and effects on future requests differ.
Illustrative example: Selecting a reply today changes today's output; training on the pair may change tomorrow's policy.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Basics
Which human work does RLAIF replace?
Show answer and example
Answer: AI compares candidate preferences to reduce pairwise human labeling and supplies strategy training signals. Evaluation criteria and independent assessment remain. AI labels can have position, style and knowledge biases.
Illustrative example: People can specify criteria while AI compares summaries; its scores aren't objective truth.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Core mechanism
Why does conversation programming need explicit termination instead of endless role replies?
Show answer and example
Answer: Messages can keep triggering responses. Without responsibilities, tool boundaries, and stopping rules, collaboration can repeat or loop. Conversation programming composes these controls for group and nested chats. More roles do not guarantee effective division; completion still needs verified artifacts.
Illustrative example: Example: A writer and reviewer repeatedly suggest improvements. Define a stop after verification passes or budget ends.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Implementation details
Does the local planner/executor/critic protocol run the full AutoGen runtime?
Show answer and example
Answer: No. It implements deterministic handoff, message logs, and stopping without full runtime, real LLM group chat, or container execution. Round counts verify the protocol; perfect scores are not the paper's multi-application ability or development efficiency.
Illustrative example: Example: One rule-based message per role can pass a protocol without solving an open software task.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Method comparison
How does message-driven orchestration differ from a fixed linear script?
Show answer and example
Answer: A linear script predefines one order. Message policies choose recipients, nested conversations, and stopping based on state or content. Flexibility helps tools and humans collaborate but can add useless chatter and loops. More messages or roles do not prove gains; count their costs.
Illustrative example: Example: A failed review can trigger one nested repair, while a pass ends immediately instead of always running five turns.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Basics
Why does AutoGen organize multi-agent applications as programmable conversations?
Show answer and example
Answer: Models, execution tools, and people may need repeated feedback beyond one question and answer. Configurable roles and messages let tool results or replies trigger later steps. Role names are not collaboration ability; specify responders, handoff, stop conditions, and actual artifact checks.
Illustrative example: Example: An assistant proposes code, an executor returns an error, and the assistant revises under a protocol rather than endless chat.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Core mechanism
Why can structured artifacts reduce hallucination cascades compared with free role chat?
Show answer and example
Answer: Downstream roles consume requirements, designs, and test contracts whose fields and constraints can be checked instead of trusting verbal promises. SOPs define order and dependencies so faults are easier to locate. Valid format does not ensure correct content or usable software; verify and execute.
Illustrative example: Example: An architect's interface lets engineers check signatures. 'The design is done' lets every role imagine a different contract.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Implementation details
How does the local temporary-repository test differ from a full MetaGPT LLM team?
Show answer and example
Answer: Four roles leave structured artifacts; engineering actually edits Python and QA runs fixed unittest. Full LLM collaboration and official SWE-bench are absent. This supports an auditable execution path, not HumanEval, MBPP, or real team delivery rates.
Illustrative example: Example: A file hash and test output verify one change and run, not autonomous architecture for unknown requirements.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Method comparison
How is PM, Architect, Engineer, and QA specialization different from repeated self-paraphrase?
Show answer and example
Answer: Each stage has distinct inputs, outputs, and artifacts: requirements, design, implementation, and tests. Gains depend on contracts and feedback, not labels that merely sound independent. Shared wrong assumptions can still break every stage.
Illustrative example: Example: QA should run tests rather than rephrase the engineer's claim that the code looks correct.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Basics
Why does MetaGPT organize agents around software roles and deliverables?
Show answer and example
Answer: Requirements, design, implementation, and testing depend on each other. Free chat can distort handoffs. MetaGPT gives roles explicit responsibilities and checkable artifacts. Organization helps but complete documents are not correct code; run verification.
Illustrative example: Example: An architect defines the interface, engineers implement it, and testers check it. Chat-only definitions can lead to inconsistent parameters.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Core mechanism
Why does same-prefix teacher scoring reach student mistakes better than complete teacher answers?
Show answer and example
Answer: Fixed teacher trajectories avoid many student errors. Teacher distributions on sampled student prefixes reach those states; data fraction controls fixed/on-policy mixture. Coverage doesn't guarantee teacher recovery or determine divergence choice.
Illustrative example: A correct demonstration never visits the student's wrong step, while prefix scoring can help from that actual state.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Implementation details
Why specify which distribution comes first in GKD KL?
Show answer and example
Answer: Forward/reverse KL and generalized JSD differ in coverage and concentration. Freeze teacher on identical student prefixes. Candidate teacher distributions aren't real T5 translation/summarization.
Illustrative example: KL(T||S) and KL(S||T) yield different gradients despite both being KL.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Method comparison
Is GKD student data fraction the same as IPD token mixing?
Show answer and example
Answer: No. GKD controls training-source mix of fixed and student traces. IPD mixes policies at each state. Dataset mixtures and one-step mixtures generally visit different trajectories.
Illustrative example: Half the batch from fixed answers is not choosing teacher with 50% probability at every token.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Basics
Why does GKD ask a teacher to score student-generated prefixes?
Show answer and example
Answer: Teacher paths can be idealized and miss visited errors. Sample student trajectories and ask for full teacher distributions at those prefixes, teaching recovery without only copying demonstrations.
Illustrative example: Teacher can guide next actions after the student's second step goes wrong.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
Why does KAR generate user preference knowledge separately from item facts before applying a hybrid-expert adapter?
Towards Open-World Recommendation with Knowledge Augmentation from Large Language Models
Show answer and example
Answer: User knowledge describes why someone might like an item; item knowledge describes the item's attributes. Their sources and meanings differ, so plain concatenation may not fit an ID model. Shared and specialized experts compress and align the knowledge for prediction with the original representations. Offline caching avoids online autoregressive generation, but LLM inferences can be wrong and are not user statements.
Illustrative example: Illustration: Waterproof material is an item fact; frequently viewing rainy-day hiking content suggests a user preference. An adapter can match them without claiming the user explicitly wants those shoes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
Why validate KAR's four-expert gate together with the original ID representations?
Towards Open-World Recommendation with Knowledge Augmentation from Large Language Models
Show answer and example
Answer: The gate mixes knowledge vectors from shared and specialized experts, then the result is scored with user and item IDs. Dimensions, identities, and gradient flow must agree. Pre-generating a knowledge file does not prove the ranker learned to use it. Cache only from training-visible behavior so preferences do not absorb test answers.
Illustrative example: Illustration: A ranker can output scores while never reading the knowledge on disk, but that is not KAR. Summarizing user preferences after seeing test behavior instead leaks the target into evaluation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
How do cost and error propagation differ between KAR's offline knowledge enhancement and online LLM recommendation?
Towards Open-World Recommendation with Knowledge Augmentation from Large Language Models
Show answer and example
Answer: Offline generation reuses a cache, so serving runs only the adapter and original recommender without autoregressive latency. Mistakes are also reused, making versions and refreshes important. Online generation is more flexible but incurs a cost on every request. The twenty-user reduction in the source shows only a small trend, not general industrial gains.
Illustrative example: Illustration: One cached film fact can serve many users, but an incorrect year propagates until refreshed. Asking on each request can update it at the cost of extra latency and inconsistency.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why separate user preference knowledge and item facts in KAR?
Towards Open-World Recommendation with Knowledge Augmentation from Large Language Models
Show answer and example
Answer: A preference explains why a user might like something, while a fact describes what it is; evidence and update frequencies differ. Factorization prompting generates them separately, and the hybrid-expert adapter injects them into the ID recommender rather than conflating facts and guesses in one text. Generated knowledge can still hallucinate and is not observed ground truth.
Illustrative example: Illustration: Enjoying light plots is an inferred preference; a film's release year is factual. Writing the preference as an item attribute can contaminate matching and explanations.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Core mechanism
How does reverse KL affect small students' treatment of teacher tails?
Show answer and example
Answer: Student expectation focuses capacity on visited teacher-supported modes; forward KL covers teacher mass. Direction changes gradients and expectation. Preserve teacher-mixed stabilization; candidate checks aren't long-text distillation.
Illustrative example: A limited student may concentrate on a few reliable phrasings rather than cover every rare teacher expression.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Implementation details
Whose distribution defines MiniLLM reverse-KL expectation?
Show answer and example
Answer: Student, with explicit teacher-mixed sampling and corrections in practice. Arbitrary fixed teacher text is not a substitute. Local candidates lack real tokenizer, long generation and GPT judging.
Illustrative example: Scoring teacher solutions with a KL-looking formula doesn't provide student on-policy coverage.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Method comparison
What coverage tradeoff separates MiniLLM from forward KL?
Show answer and example
Answer: Forward emphasizes teacher-mass coverage; reverse follows student-sampled modes. Teacher-mixed sampling and variance reduction also matter; direction alone isn't the full method.
Illustrative example: Few reliable modes versus many weak modes requires task evaluation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Trajectory distillation: students, proposals, and prefix distributions
Basics
Why favor reverse KL in MiniLLM?
Show answer and example
Answer: Small students may not cover all teacher modes. Reverse KL concentrates on student-supported reliable teacher regions, possibly missing modes. Sharpness isn't always desirable.
Illustrative example: Learn several reliable phrasings rather than spread over all teacher continuations.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Core mechanism
Why use equivalent scaling in AWQ rather than separately storing a few higher-precision weights?
AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
Show answer and example
Answer: Calibration activations identify output-sensitive input channels. Reciprocal input/weight scaling preserves the float function while making those weights easier to represent in one low-bit format. Large weights alone do not determine sensitivity, and KV growth is unrelated.
Illustrative example: Equal weight errors have different output effects on channels with small versus large activations.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Implementation details
Must input be inversely scaled when weights are scaled by S?
AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
Show answer and example
Answer: Yes: XW=(XS⁻¹)(SW). Without reciprocal scaling the floating function already changes. Search scales by quantized reconstruction error; local MSE does not establish TinyChat throughput.
Illustrative example: Multiply one weight channel by two and divide its input by two.
Sections in the Chinese source: 核心公式, 复现边界. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Method comparison
What does AWQ add over round-to-nearest W4?
AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
Show answer and example
Answer: Search activation-aware equivalent channel scales before W4 rounding. RTN uses the given scale and nearest grid. Both can retain a uniform low-bit format.
Illustrative example: Rescale an important channel before rounding rather than round original weights directly.
Sections in the Chinese source: 背景与主要改动, 本地复现. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Basics
Why use activations instead of weight magnitude alone?
AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
Show answer and example
Answer: Output error depends on input activation. Calibration identifies salient channels and equivalent scaling protects them before uniform low-bit weight-only quantization.
Illustrative example: The same weight error is more consequential when multiplied by large activations.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Core mechanism
Why actively label uncertain steps instead of only checking final answers?
Show answer and example
Answer: Final outcomes don't locate early mistakes. Step labels provide detailed reward-model supervision; uncertainty targets current ambiguous decisions but isn't a universal error proxy. Control candidate/evaluation budgets; no full local annotation dataset.
Illustrative example: Review an uncertain key algebraic move rather than repeatedly labeling obvious format.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Implementation details
How separate PRM labels and predictions?
Show answer and example
Answer: Step truth is fixed supervision, probabilities differentiable, and entropy used only in allowed acquisition. Test step labels cannot select training samples. Candidate budget tests mechanism only.
Illustrative example: Training on observed test-label errors leaks evaluation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Method comparison
What tradeoff separates active uncertain-step labeling from random labeling?
Show answer and example
Answer: Active selection spends budget on hard judgments; random selection better reflects the original distribution. Uncertainty isn't importance and selection bias/label quality remain.
Illustrative example: An ambiguous formatting step can be less valuable to review than a key mathematical transition.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Reward and credit: which steps deserve a good final score?
Basics
Why do process supervision methods label steps rather than only outcomes?
Show answer and example
Answer: Failure doesn't identify the broken step. Local labels provide finer learning and selection, requiring annotations or reliable process judges. Do not broadcast terminal correctness as step truth.
Illustrative example: If step three fails, steps one and two needn't be marked wrong.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Core mechanism
Why use DPO policy changes relative to reference instead of raw answer probabilities?
Show answer and example
Answer: Raw scores inherit pretraining preferences. Relative ratios measure redistribution against the baseline, jointly comparing preferred/rejected in the classification margin implied by KL regularization. No explicit reward model, but same-prompt preferences and fixed reference remain.
Illustrative example: If a phrase starts with high likelihood, distinguish inherited preference from a further supervised increase.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Implementation details
Why freeze DPO's reference and align response masks?
Show answer and example
Answer: The reference anchors changes; updating it changes the objective. Score identical response tokens in both paths, with neither prompt nor padding mismatches.
Illustrative example: Adding prompt likelihood only to reference introduces irrelevant input length into the margin.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Method comparison
How does DPO's data flow differ from reward-model PPO?
Show answer and example
Answer: Classic DPO consumes fixed preference pairs without sampling and reward-model scoring at every step. PPO usually includes those stages. A fixed dataset may omit states future policies visit.
Illustrative example: A preference bank without a wrong-prefix category doesn't automatically gain it from offline updates.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Basics
Why doesn't DPO need a separately fitted reward model?
Show answer and example
Answer: The KL-regularized policy expresses implicit reward as a policy/reference log ratio. Direct paired classification uses chosen and rejected labels. Reward information still comes from preference supervision.
Illustrative example: With labels on two answers, compute the paired loss without first training a scorer.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Core mechanism
How can a planner express tool dependencies without rereading intermediate observations?
Show answer and example
Answer: The planner writes dependencies as variable references. The Worker substitutes obtained evidence into later tool arguments, and the Solver combines results at the end. This reduces repeated planning calls without deleting data flow. A static plan is less adaptive if an unexpected observation determines a later route that was never encoded.
Illustrative example: Example: Store a person's birthplace as E1, then look up its population using E1. The city is bound later without regenerating the whole plan.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Implementation details
When can a Worker run tools in parallel, and when must it wait for an evidence variable?
Show answer and example
Answer: Calls without unmet dependencies can run in parallel. A call referencing an earlier E variable must wait for that result. The Solver reads both the plan and actual evidence at the end, not imagined planner results. Local tests verify control flow and counts without real search APIs or a multi-size LLM comparison.
Illustrative example: Example: Weather requests for two known cities can run together. Weather for the newly found city must wait for lookup rather than guess its argument.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Method comparison
What do ReWOO and ReAct trade off when handling unexpected intermediate results?
Show answer and example
Answer: ReAct reasons after each observation and can revise its route, but repeated model calls cost more. ReWOO plans before execution and avoids repeatedly reading evidence, but relies more on a workable initial dependency plan. Unexpected errors require separate recovery; a static plan is not inherently a feedback loop.
Illustrative example: Example: When the first query returns several namesakes, ReAct can add disambiguation immediately. ReWOO needs an outer recovery if its original plan has no such branch.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Basics
Why can a multi-step tool task spend heavily on rereading existing content?
Show answer and example
Answer: If every return makes the planner reread the task, old plan, and all evidence, input grows even when the next action only substitutes a known result. ReWOO separates planning, tool execution, and final synthesis to reduce repetition. It fits workflows whose dependencies can be expressed in advance, while surprises still need recovery.
Illustrative example: Example: Finding an author and then their birthplace is a known two-step pattern. Each returned string does not require regenerating the plan and rereading all earlier evidence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why admit only programs verified by the environment to Voyager's skill library?
Show answer and example
Answer: Plausible code can fail because materials are missing, APIs are wrong, or goals are unmet. Environment feedback exposes these failures. Index confirmed successes for reuse. The automatic curriculum picks new goals and the library retains verified behavior; code generation alone cannot substitute.
Illustrative example: Example: A program for crafting an iron pickaxe is not a successful skill if inventory lacks iron. Only actual completion supports future reuse.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
Is the local structured-plan skill library equivalent to Mineflayer JavaScript programs?
Show answer and example
Answer: No. Curriculum, skills, verification, and cross-episode reuse are retained with structured tool plans. The run omits Minecraft, GPT-4, and embedding retrieval. Controlled retry counts do not represent real program debugging or paper item and distance gains.
Illustrative example: Example: Reusing a plan template does not show handling Mineflayer's asynchronous movement errors or achieving its technology-tree gains in a new world.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
How does automatic curriculum differ from a fixed checklist for accumulating skills?
Show answer and example
Answer: The curriculum chooses novel goals from current state and skills. A fixed list is controlled but cannot adapt to capability. Self-selected goals can favor easy novelty, so validate outcomes and progress rather than treat task count or exploration distance as general capability.
Illustrative example: Example: After acquiring stone tools, mining explores more than repeated wood gathering, but completion still depends on inventory and milestones.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
Why does Voyager use code as an action instead of only a natural-language plan?
Show answer and example
Answer: Code invokes environment APIs over extended behavior and can be iteratively repaired from errors and observations. Successful programs enter the library for reuse. Natural-language plans still need executable mapping. Runnable code can also be wrong and needs environmental validation.
Illustrative example: Example: Compile collecting wood into movement, search, and harvesting loops, then inspect inventory. Saying 'chop a tree' does not perform those actions.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Core mechanism
Why constrain format and check real execution in addition to retrieving usage examples?
Show answer and example
Answer: Examples show when and how to call tools but cannot guarantee schema or argument meaning. Format constraints ensure parseability; execution tests whether the goal is achieved. Experience, contracts, and observed results are distinct. JSON-like text is not actual tool capability.
Illustrative example: Example: A complete weather-call schema can still name the wrong city. Valid JSON does not answer the requested forecast.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Implementation details
Does local planbench-mini tool success equal the paper's eight tasks or live API success?
Show answer and example
Answer: No. Local work tests demonstration, constraint, or state protocols without paper-scale models, training, or the full real tool matrix. Tool results and scoring require independent execution. Perfect controlled results are not public benchmark scores, and similarly named ToolBench datasets must not be conflated.
Illustrative example: Example: A preset plan can parse and execute without proving the model can call an unseen service from documentation.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Method comparison
How does retrieving in-context demonstrations differ from always using one fixed example set?
Show answer and example
Answer: Retrieval chooses examples for the current task, avoiding irrelevant context and broader coverage gaps of fixed examples. It can still choose badly or encourage copied arguments. Reuse procedures, not answers. Compare fixed models and tasks and include example-reading cost.
Illustrative example: Example: SQL examples help database queries more than weather samples, but their table names cannot be reused unverified.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Basics
Why teach a model multiple tools rather than handwrite calls for each ToolBench task?
Show answer and example
Answer: Real catalogs have varied interfaces and parameters. A model must understand intent, find relevant demonstrations, and generate executable calls. ToolBench studies data and constraints for transferable tool use. Format is one step; verify API behavior, outputs, and final answers rather than synthetic success alone.
Illustrative example: Example: Weather and map tools require different fields. Use each contract instead of copying one fixed template.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Core mechanism
Does GQA reduce Q heads or KV heads, and why does it save decode memory?
GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints
Show answer and example
Answer: Many query heads share fewer K/V heads while retaining different queries. Each historical position stores and reads fewer KV groups. This reduces independence compared with MHA, not history token count.
Illustrative example: Eight queries can use two KV groups while producing eight different attention distributions.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Implementation details
Does temporarily expanding shared KV establish cache savings?
GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints
Show answer and example
Answer: No. Logical broadcast can aid compute, but persistent KV must store fewer groups; retaining expanded copies loses savings. Check g(h) and distinguish local from original uptraining.
Illustrative example: Four Q heads sharing two groups should store two histories.
Sections in the Chinese source: 核心公式, 复现边界. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Method comparison
Which head-count axis distinguishes MHA, GQA, and MQA?
GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints
Show answer and example
Answer: MHA generally has one KV per Q, GQA shares within groups, and MQA shares globally. Query head count can remain fixed; quality and bandwidth are empirical.
Illustrative example: Keep eight Q heads with eight, two, or one KV group.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Inference compression: weights, KV cache, and serving bandwidth
Basics
Why share fewer K/V heads among more queries?
GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints
Show answer and example
Answer: Queries keep distinct directions while grouped history lowers cache and read bandwidth. It interpolates between per-head MHA and single-group MQA.
Illustrative example: Four query heads can share two stored KV histories instead of four.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Core mechanism
Why can external tool feedback reveal persistent errors that rereading an answer misses?
Show answer and example
Answer: Self-critique may inherit the original model's assumptions. Search facts or code outputs provide independently observable constraints. Evidence guides revision in a loop. Tools can fail or return nothing; no feedback is not verification, and success must not be fabricated.
Illustrative example: Example: A wrong product may still look right to the model. A calculator result provides a concrete contradiction for revision.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Implementation details
What feedback does the local real-traceback loop cover?
Show answer and example
Answer: It writes a defective patch, runs unittest, edits from observed feedback, reruns, and records commands and exit codes each round. This verifies controlled code feedback and iteration, not search engines, toxicity classifiers, or a general LLM critic. Running tests does not validate all paper tasks.
Illustrative example: Example: A second patch turning a failed test into a pass verifies this code instance, not open-domain QA or harmlessness.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Method comparison
What evidence does code execution provide for critique versus web retrieval?
Show answer and example
Answer: Execution shows outputs or exceptions for a specific input. Web retrieval supplies external text whose source and support must be assessed. Both extend self-evaluation, with different limits: untested code can still fail, and a search hit need not support the answer.
Illustrative example: Example: Division by zero proves one run failed. A related web result still needs its actual passage read, not just its title.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Basics
Why does CRITIC try to find and fix errors after the model has already written an answer?
Show answer and example
Answer: Drafts can contain factual, reasoning, or code errors despite confidence. CRITIC gathers external checks for critique and revision. Observable evidence helps, but tools have limited coverage and edits can add errors. Evaluate checks and final outcomes separately.
Illustrative example: Example: Execute code on a boundary input, observe an exception, and revise. Rereading the same explanation gives no equivalent evidence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Core mechanism
Why keep supervised cross-entropy when SLiC-HF already has a preference margin?
Show answer and example
Answer: Relative ranking does not prevent overall language drift. Cross-entropy preserves reference response likelihood, combining calibration and generation learning. Off-policy data do not remove coverage, length or source differences.
Illustrative example: Suppressing bad answers can satisfy ranking even while good answers get worse; supervision retains good generation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Implementation details
Why can SLiC-HF use pairs collected by another model?
Show answer and example
Answer: The objective can score fixed responses and doesn't need current-policy samples. Keep sources and same-prompt pairing. Local candidate calculations are not the paper's T5 and human evaluation.
Illustrative example: Old summaries can be scored by the current policy if labels still belong to those summaries.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Method comparison
How does SLiC-HF's hinge margin differ from DPO's reference-relative one?
Show answer and example
Answer: The hinge directly constrains sequence likelihood differences alongside supervision. DPO contrasts log ratios relative to a frozen policy. Both baseline and loss shape differ.
Illustrative example: Once the hinge margin is met, ranking penalty can stop; reference-relative preference is different.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Basics
Why combine sequence calibration and supervised loss in SLiC-HF?
Show answer and example
Answer: Ranking makes preferred likelihood exceed rejected likelihood. Supervision learns trusted content rather than succeeding only by suppressing bad replies. Both train the generator directly.
Illustrative example: Separate good from bad summaries while learning the good summary itself.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Core mechanism
Why emphasize excess loss rather than maximum raw loss in DoReMi?
DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretraining
Show answer and example
Answer: Raw loss mixes irreducible difficulty and underfitting. Subtract reference loss and use group DRO on a small proxy to derive target mixture. This is relative to the reference and data, not a preference for all high-entropy domains.
Illustrative example: A has loss 5 vs reference 4.9; B has 3 vs 2. B's larger excess can get more weight.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Implementation details
Why normalize α across domains after each update?
DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretraining
Show answer and example
Answer: Exponential updates yield relative weights, not probabilities; normalization sums to one. Keep the same reference. Local two-domain unigram work is not a Pile 280M-to-8B transfer.
Illustrative example: Relative weights 2 and 3 become 0.4 and 0.6.
Sections in the Chinese source: 核心公式, 复现边界. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Method comparison
How do dynamic DoReMi weights differ from uniform sampling?
DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretraining
Show answer and example
Answer: Uniform probabilities stay fixed. DoReMi exponentiates reference-relative excess-loss updates and renormalizes, responding to underfitting rather than raw loss alone.
Illustrative example: A high loss from an intrinsically hard domain may not imply a large excess.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Training budgets: data mixtures, objectives, and optimizers
Basics
Why learn target-model mixtures with a small proxy?
DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretraining
Show answer and example
Answer: Repeated large runs are costly. Small-proxy group DRO adjusts weights from excess loss, then target training uses that mixture. Transfer needs verification.
Illustrative example: Estimate reference-relative gaps cheaply before allocating large-run data.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Core mechanism
How does searching a tree of thoughts differ from generating several complete answers at once?
Show answer and example
Answer: Intermediate thoughts are states that can be evaluated and expanded separately. Search can retain early alternatives, prune dead ends, and backtrack. Complete answers are often compared only at the end, paying repeatedly for bad prefixes. ToT still depends on value estimation and width/depth budgets; a tree does not make its evaluator correct.
Illustrative example: Example: An arithmetic problem has three possible initial decompositions. Check which preserve feasible remaining values before expanding, rather than finishing a full answer for obviously impossible ones.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Implementation details
Does the local workflow-coherence value represent GPT-4 self-evaluation quality?
Show answer and example
Answer: No. A small PlanBench uses deterministic workflow coherence, a fixed beam, and backtracking to test thought-tree control flow, not GPT-4 or Game of 24. Counts of nodes and backtracks show that search happens. A perfect score does not establish an open-ended language value function.
Illustrative example: Example: A rule can flag an action that violates a precondition. That does not prove a large model can reliably score the future promise of a creative-writing fragment.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Method comparison
How do BFS and DFS spend limited ToT search budgets?
Show answer and example
Answer: BFS retains multiple states at one layer, making it easy to compare alternatives at similar progress but requiring a broad evaluated frontier. DFS goes deep on one path and backtracks, keeping a narrower frontier but potentially wasting depth on a bad branch. Width, depth, and pruning thresholds depend on evaluation reliability.
Illustrative example: Example: With three opening moves, BFS explores one step from each. DFS may follow the first to the end before trying the second.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Basics
Why is ToT suited to reasoning that requires trying paths and backing out of dead ends?
Show answer and example
Answer: Local choices affect later feasibility. One uninterrupted answer can carry a bad choice to the end. ToT represents expandable intermediate states, retaining directions, judging progress, and backtracking. This requires states that can be meaningfully evaluated; search does not supply a reliable judgment rule by itself.
Illustrative example: Example: If a partial expression cannot reach the target number, change the previous grouping instead of regenerating the entire solution.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Core mechanism
Why share semantics via residual SID instead of random hierarchy, and still use a collision token?
Recommender Systems with Generative Retrieval
Show answer and example
Answer: Coarse centers and residual refinement can group related items, while finite codebooks still collide. A unique suffix resolves identity. Matched random IDs isolate hierarchy from semantic hierarchy; semantic IDs need not win.
Illustrative example: Gardening books can share a prefix and split by subject or final collision suffix.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Implementation details
Does local TIGER rank with a score proxy or complete code probability?
Recommender Systems with Generative Retrieval
Show answer and example
Answer: Sum token log probabilities over valid item codes, without score fusion. Match codebooks/collisions in training and decoding; exclude invalid paths.
Illustrative example: Omitting a level changes the complete item's generation probability.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Method comparison
How does a collision token differ from the semantic prefix?
Recommender Systems with Generative Retrieval
Show answer and example
Answer: It disambiguates item identity when several items share a semantic code. An arbitrary unique suffix is not a new semantic level.
Illustrative example: Two similar videos can share three levels but require distinct final tokens.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Generative retrieval and reasoning-based recommendation
Basics
Why quantize content into multilevel Semantic IDs?
Recommender Systems with Generative Retrieval
Show answer and example
Answer: Short discrete codes share meaning and support autoregressive generation. Content and quantization quality limit generalization, and codes must map to valid items.
Illustrative example: Similar items share early codes and differ at fine levels.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Core mechanism
Why is projecting visual features to the LLM width insufficient for following visual instructions?
Show answer and example
Answer: Matching shape does not align meaning. The connector learns a visual-to-language interface, and instruction supervision teaches how to answer questions conditioned on images. Projection alone does not confer recognition, localization, or multiround responses.
Illustrative example: A cat image can enter as a vector, but QA training is needed to answer “Where is it?”
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Implementation details
Does the local two-layer projector's negative result refute LLaVA's instruction tuning?
Show answer and example
Answer: No. The local run fixes a small patch encoder, classification head, and budget, and compares a linear projector with a two-layer MLP. It does not generate GPT-4 instructions or train LLaVA-13B. Extra connector capacity did not help this configuration, not all tasks.
Illustrative example: A decline in small clothing classification says nothing directly about large-model multiround image dialogue.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Method comparison
How does LLaVA instruction supervision differ from CLIP's dual-encoder contrastive training?
Show answer and example
Answer: LLaVA learns conditional answer tokens from images and questions. CLIP brings paired images and texts together in a shared embedding space for matching. Retrieval scores are not instruction-generation quality, and architecture alone does not establish superiority.
Illustrative example: Answering why two people hold umbrellas is conditional generation; selecting a supplied matching caption is a different output contract.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Basics
What does LLaVA's visual projector do between image and answer?
Show answer and example
Answer: It maps image-encoder features into visual tokens that the language model reads alongside questions and conversation history. It connects the interfaces; multimodal instruction training, not shape alone, teaches instruction following.
Illustrative example: A width-matching projection maps encoder features to LM size, but the model still must learn which features address the user's question.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Core mechanism
How does reward selection followed by maximum likelihood differ from direct policy gradients?
Show answer and example
Answer: Rank sampled replies with the reward model and supervise selected text; scores determine selection, not per-action gradient weights. Resample with the next policy. Judge bias and missing candidates persist; local tests cover the iteration state.
Illustrative example: Learn the highest-rated candidate summary; absent good summaries can't be chosen and judge errors can be copied.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Implementation details
How isolate RAFT selection from evaluation?
Show answer and example
Answer: Log reward model and rollout version, select among same-prompt candidates then update policy. No test answers for sorting. Local candidates aren't a real generator or reward model.
Illustrative example: Training on choices made with test correctness invalidates held-out gains.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Method comparison
How does reward enter RAFT differently from PPO?
Show answer and example
Answer: RAFT uses reward to select texts and then applies maximum likelihood. PPO uses advantages and probability ratios directly in policy objective. Selection and continuous weighting differ.
Illustrative example: A high score can decide archive admission without proportionally increasing CE.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Data and budgets: where to spend the next unit of compute
Basics
Why keep generating fresh RAFT candidates from current policy?
Show answer and example
Answer: Fixed SFT data can't follow improvement. Resample, rank by reward and supervise better behavior next round. Coverage still limits which outputs can be chosen.
Illustrative example: Improved summarization can generate better candidates for later training.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Core mechanism
Why still apply SFT to the best answer after ranking responses?
Show answer and example
Answer: The hinge only fixes ordering conflicts and can stop giving a useful generation signal once sorted. SFT raises high-quality content likelihood. External rewards and length normalization do not guarantee factual correctness.
Illustrative example: Even with three answers ranked correctly, the best may be rare; its SFT term can still increase likelihood.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Implementation details
Why isn't RRHF p_i raw probability?
Show answer and example
Answer: It is length-normalized response log probability. The hinge compares this order within one prompt with consistent reward direction. Candidate tests do not reproduce full 7B training.
Illustrative example: Using decimal probabilities in place of mean logp changes margins and gradients.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Method comparison
What do best-response SFT and RRHF ranking each add?
Show answer and example
Answer: Ranking corrects relative order; SFT directly raises best-response likelihood. Ranking alone can succeed by suppressing alternatives. The combination isn't a universal optimum.
Illustrative example: A good answer should be learned, not merely score above a bad one.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Basics
Why use a full ordering of multiple same-prompt answers in RRHF?
Show answer and example
Answer: Multi-source responses provide more relative comparisons than one pair. Match length-averaged log likelihood to reward order and supervise the best answer. Ranking quality depends on labels/reward.
Illustrative example: Good, middling and poor responses supply several orderings, not one winner pair.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Core mechanism
Why shouldn't an important old event automatically replace more query-relevant memory?
Show answer and example
Answer: Importance captures long-term significance, relevance measures current fit, and recency models decay. Reflection abstracts several important events, but actions still need current state. Using one dimension alone can repeat major events or focus only on recent chatter.
Illustrative example: Example: Moving home was important, but last night's restaurant confirmation better answers today's dinner-location question.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Implementation details
Do local reflection triggers include a social evaluation of 25 LLM personas?
Show answer and example
Answer: No. Controlled EvoMem tests three-score retrieval, cumulative-importance triggers, and planning dependencies without a Smallville map, real personas, or human credibility judgments. Reflection counts and proxy costs show mechanism execution, not believable relationships or coordination.
Illustrative example: Example: A threshold producing one rule-based reflection does not show models spreading invitations or arranging a party.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Method comparison
What questions do reflection and planning answer in everyday behavior simulation?
Show answer and example
Answer: Reflection forms higher-level interpretations from experiences, such as relationship or habit inferences. Planning schedules what to do next using goals, memory, and time. Inferences are not events, and plans are not executed results. Separation makes both evidence and actual behavior traceable.
Illustrative example: Example: Inferring that a neighbor likes gardening is reflection. Deciding to invite them to buy seeds this afternoon is planning.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Memory and skills: retrieval, reuse, and invalidation
Basics
How does reflection differ from recording a single observation?
Show answer and example
Answer: Observations are concrete events. Reflection draws higher-level inferences across important memories for retrieval and planning. Inferences can be wrong and are not primary evidence. Keep observations traceable and plans grounded in current state.
Illustrative example: Example: Several helpful acts may suggest someone is cooperative. That is an abstraction, not a new observed event, and should retain support.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Core mechanism
Why is inception prompting with roles, goals, and boundaries more than naming two models?
Show answer and example
Answer: Prompts define responsibilities, permitted requests, and cooperation. Alternating messages advance a common task under constraints. Renaming without capability or boundary differences can produce duplicates rather than collaboration. A social trace does not prove goal achievement; check deliverables separately.
Illustrative example: Example: One role specifies implementation and the other proposes solutions without changing the overall goal. Mutual praise does not move the task forward.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Implementation details
Does the local collaboration state machine match the full paper's multi-agent instruction-following scope?
Show answer and example
Answer: No. It tests role messages, state updates, and budget without full large-model social traces or benchmarks. Scripted lines can verify handoffs but not autonomous cooperation or general open-task success.
Illustrative example: Example: Two templated replies show alternation works, not that models maintain role and goal boundaries on unfamiliar tasks.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Method comparison
What does alternating two-role cooperation trade off against one complete model response?
Show answer and example
Answer: Dialogue can clarify and divide reasoning, but adds turns, communication, and error propagation. Models sharing biases cannot validate one another by agreement. Evaluate task outcomes and total cost rather than natural-sounding social messages.
Illustrative example: Example: An executor can ask for missing input. If both roles misread the goal, more turns only extend the mistake.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Basics
Why does CAMEL study role collaboration instead of only a single model answer?
Show answer and example
Answer: Some tasks need requests, actions, and feedback over time. Two constrained roles can produce a longer collaboration process and data. Dialogue length and role division do not prove quality; roles can misunderstand goals, repeat, or drift. Inspect outcomes.
Illustrative example: Example: One role requests analysis and another implements with feedback. Repeating the request across turns does not complete the analysis.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Core mechanism
Why generate explicit feedback separately when the same model writes both draft and feedback?
Show answer and example
Answer: Explicit feedback converts unmet constraints into actionable edits, avoiding mere random resampling. Generation, feedback, and refinement play distinct roles, and stopping still depends on the model. No extra training does not make feedback correct, and repeated polishing is not independent fact-checking.
Illustrative example: A draft omits units. Feedback to add units to each number gives a concrete target that 'write it better' does not.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Implementation details
Does stopping under a deterministic local critic demonstrate reliable real-LLM self-evaluation?
Show answer and example
Answer: No. A controlled critic verifies generation, feedback, revision, and stopping states, without a general LLM or the seven original benchmarks. Stopping verifies the contract, not reliable recognition of factual correctness on open tasks. Local perfect scores cannot stand in for human preferences.
Illustrative example: A rule can stop once text length is sufficient even when facts are wrong. Real self-assessment can be satisfied too early and needs independent evidence.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Method comparison
Compared with Reflexion's cross-trial episodic memory, what loop does Self-Refine emphasize?
Show answer and example
Answer: Self-Refine revises the current draft through feedback. Reflexion saves failure experience in bounded memory for a later trial. They can be combined but use different state. Refinement does not automatically form long-term memory, and repeated rewriting is not parameter RL.
Illustrative example: Revising one email three times is Self-Refine. Saving 'check recipients first' for tomorrow's email involves cross-trial memory.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Basics
Why treat an output as a draft that can be improved against specific requirements in Self-Refine?
Show answer and example
Answer: Open-ended generation may require correctness, completeness, clarity, and style; one attempt may meet only some. Self-Refine generates, diagnoses shortcomings, and revises without extra parameter training. The loop needs actionable feedback, and mistaken feedback can make later drafts worse; more rounds do not imply better quality.
Illustrative example: A correct first explanation lacks an example. Point out the gap before revising; 'make it better' may not identify what to change.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Core mechanism
Why obey subtask dependencies instead of calling all multimodal experts simultaneously?
Show answer and example
Answer: A later expert may require actual outputs from detection, recognition, or transcription. Build a DAG, select compatible experts, run in topological order, then synthesize. Similar names do not imply compatible interfaces. Parallelize independent work, not dependencies.
Illustrative example: Example: Translate only after receiving real speech transcription. Starting translation concurrently and guessing the transcript breaks provenance.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Implementation details
Do the local capability registry and dependency edges run downloaded expert models?
Show answer and example
Answer: No. Local tool descriptions replace the model catalog to check matching and topological execution. No hundreds of Hugging Face checkpoints or ChatGPT calls ran. Edge counts verify flow, not cross-modal quality or real costs.
Illustrative example: Example: A detection node's rule-generated output can reach the answer node without establishing actual vision recognition.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Method comparison
What does the language controller do versus a specialist multimodal model?
Show answer and example
Answer: The controller interprets requests, selects specialists, and schedules subtasks. Specialists process vision or audio and the controller synthesizes results. Correct routing does not ensure correct predictions, and individual predictions do not guarantee a correct final answer. Verify selection, I/O, and interpretation separately.
Illustrative example: Example: A controller schedules image recognition before speech generation. The image and speech models execute their roles; the controller does not acquire their parameters.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Basics
Why does HuggingGPT use expert model descriptions as a searchable capability catalog?
Show answer and example
Answer: The controller matches subtasks to model-card descriptions and invokes experts instead of implementing all vision and speech itself. Descriptions guide selection, not guarantee capability. Verify versions, input/output types, and availability.
Illustrative example: Example: Object recognition calls detection; audio-to-text calls ASR. Their media and output contracts differ.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Core mechanism
Why is verbal reinforcement not reinforcement-learning updates to model parameters?
Show answer and example
Answer: Feedback becomes natural-language experience in bounded episodic memory. The next attempt sees a different context without gradient updates to weights. Reflection can communicate more than binary success but can be wrong. An Evaluator must still judge execution rather than treating reflection's claims as learning evidence.
Illustrative example: Remember 'check the file path before editing' after failure. Reading that note changes behavior next round even with identical weights.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Implementation details
Why interpret the mini-suite score of 0.9 separately from isolated ToolRoute results?
Show answer and example
Answer: The mini-suite deterministically fails the first attempt in each family and then reuses feedback, with fixed reflection templates and evaluator. ToolRoute is a different tool environment without a reference-policy interface. Neither is HumanEval pass@1. Bounded append and failure retention can be checked, but scores cannot rank across protocols or replace the paper.
Illustrative example: One deliberately failed task in each of twelve families sets the aggregate rate, unlike natural model errors across 120 independent coding problems.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Method comparison
What does actionable reflection add over saving a binary failure reward?
Show answer and example
Answer: Binary reward only says the attempt failed. Reflection proposes a cause and actions to try or avoid, supplying actionable context. That inference may be wrong and needs actual next-trial feedback. Do not place the hidden reference plan into reflection and credit later success to autonomous learning.
Illustrative example: 'Failure' does not locate the problem. 'The date parameter needs ISO format' is useful, but must come from a visible error rather than a gold route.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Harnesses and self-improvement: which layer changes?
Basics
Why does Reflexion retain failure lessons between attempts rather than just resample?
Show answer and example
Answer: Repeatedly trying the same task by resampling may repeat errors. Reflexion converts execution evaluation into readable experience so feedback survives across trials. Storing a lesson does not establish accuracy or transfer. Distinguish external reward from text reflection; neither automatically updates model weights with RL.
Illustrative example: If the first search stalls on an irrelevant page, reflection can advise checking sources and expanding the query. Restarting without the lesson can follow the same route.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Core mechanism
Why pause at a tool marker and resume after execution instead of generating a whole program with results at once?
Show answer and example
Answer: Pausing conditions later text on real tool returns rather than imagined outcomes. Retrieved reasoning/tool examples reduce manual prompts, but the runner must detect markers and inject observations. Frozen weights do not prevent tool feedback changing context and future output.
Illustrative example: Example: Execute a database query at its marker, then summarize the three returned rows rather than claim ten and add an unrun query.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Implementation details
How does the local tool-signature task bank differ from program generation by a frozen LLM?
Show answer and example
Answer: Local tests check small-library retrieval, per-tool pause/resume, and updates without a frozen large model or BIG-Bench/MMLU. Pause counts show branches ran, not model-generated program semantics. Proxy costs are not paper tokens.
Illustrative example: Example: Three pauses from a structured template test the runner, not automatic programming for arbitrary natural language.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Method comparison
What is the trade-off between retrieving examples automatically and hand-writing CoT for each task?
Show answer and example
Answer: Retrieval reuses library patterns and reduces human authoring, but similar examples may miss current constraints. Human revisions can extend the library, and new programs still need verification. Copying an example does not solve a new task or eliminate maintenance.
Illustrative example: Example: Reuse search-then-compute structure, but use queries and arguments from the new problem rather than copy the old answer.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Basics
Why does ART reuse reasoning and tool demonstrations for new tasks?
Show answer and example
Answer: Handwriting full CoT and tool flows for each task is costly. Prior examples offer transferable structure. ART retrieves examples and generates workflows using external computation. Validate current inputs and actual outputs; copying old answers or assuming cross-domain transfer is invalid.
Illustrative example: Example: Reuse a calculator-based cost comparison procedure for a new bill, but obtain new numbers and execution results.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Core mechanism
Why compare later loss with the API return visible, hidden, and without a call?
Show answer and example
Answer: Call syntax alone can look useful without tool information. Counterfactual return conditions test whether actual output lowers later prediction loss. Keep calls beating the threshold for training to learn utility. This self-supervised proxy is not universal task accuracy or business gain.
Illustrative example: Example: Calculator syntax does not reveal the product. If the real return improves subsequent prediction, keep that call.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Implementation details
Why isn't local utility-loss filtering completed GPT-J continued training?
Show answer and example
Answer: Local loss comes from deterministic utility. It preserves the candidate rule without GPT-J token likelihood, 6.7B training, or five real API classes. Candidate and acceptance counts show filtering ran, not learned timing or paper zero-shot capability.
Illustrative example: Example: A preset useful tool lowering rule-based loss verifies screening, not discovery of tool value in real training text.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Method comparison
How does Toolformer construct call data differently from manually labeled tool trajectories?
Show answer and example
Answer: Human examples directly demonstrate actions. Toolformer starts with a few demonstrations, samples calls, filters by how much returns improve subsequent text prediction, then trains on retained data. This reduces per-call annotation without eliminating demonstrations, tools, or filtering requirements.
Illustrative example: Example: Instead of manually tagging each calculator call, generate candidates and filter by loss. Some bad candidates still need rejection.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Basics
Why learn when to use external tools from text training in Toolformer?
Show answer and example
Answer: Fluent models may still struggle with arithmetic, fresh facts, or retrieval. Toolformer inserts helpful API calls into training so the model learns timing, arguments, and return usage. Screening supplies learning signals, not guaranteed usefulness. A local filter does not train the model.
Illustrative example: Example: Insert a calculator call when a multiplication result helps complete the sentence, not after every number.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Core mechanism
How can Q-Former learn to connect images to a language model when both the vision encoder and language model are frozen?
Show answer and example
Answer: Gradients can still flow through frozen modules to the connector; only parameter updates are frozen. Learnable queries select visual information through cross-attention and are projected into soft prompts the language model can consume. Two training stages address vision–language representation alignment and the generation interface. The bottleneck can still lose detail.
Illustrative example: Compressing 16 image features into four queries gives the language model four visual prompts, but reading small text may need more capacity.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Implementation details
Do the local four-query results validate the paper's two-stage pretraining?
Show answer and example
Answer: No. The local work uses real Fashion-MNIST pixels and four-query cross-attention, without ViT-g, a large LM, ITC/ITM/ITG, or large-scale second-stage training. It checks token compression and small-budget behavior, not original zero-shot capability.
Illustrative example: Reducing 16 visual tokens to four proves this interface shortens the sequence, not that it pretrained on over 100 million images.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Method comparison
How does BLIP-2's interface differ from linearly projecting each visual patch?
Show answer and example
Answer: Per-patch projection primarily changes feature dimensions and usually preserves token count. Q-Former actively aggregates visual information with learnable queries before projection. This changes both selection and sequence length; lost detail depends on training and the task, not architecture alone.
Illustrative example: Clothing recognition may need only an outline, while reading small text may need local detail. Four queries are not guaranteed to suffice for both.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Basics
Are Q-Former queries the text of the question to answer?
Show answer and example
Answer: They are a fixed number of learnable vectors, not the user's question. Cross-attention reads tokens from the frozen vision encoder and passes selected information to the language side. Their fixed count forms a visual bottleneck: more complex images do not automatically produce more output tokens.
Illustrative example: Four learnable queries can attend to hundreds of image patches while sending only four vectors to the language model.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Core mechanism
Why are critique/revision and AI preference training separate stages?
Show answer and example
Answer: First critique and revise by principles, then supervise the revised texts. Next compare responses by those principles and use preference learning for RL. Demonstrations and choice labels differ. Principles and local updates do not establish harmlessness or full red teaming.
Illustrative example: Rewrite a reply to avoid privacy leakage, then compare two rewrites; text and winner labels are distinct inputs.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Implementation details
What safety claims do local Constitutional AI principles support?
Show answer and example
Answer: Deterministic axes such as correctness, format, process and brevity test principle scoring and state updates. They do not cover real harmful requests or train a complete safety reward model.
Illustrative example: Valid formatting doesn't establish reliable refusal of dangerous advice.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Method comparison
Why distinguish principle-based revision SFT from RLAIF?
Show answer and example
Answer: The former learns revised text; the latter uses AI preferences to train reward and optimize policy. Data and execution differ. Local simplification does not establish a separate preference model or PPO; arithmetic principles are not safety red-team evidence.
Illustrative example: Repairing one answer supplies a demonstration, not evidence of completed reward-driven RL.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Preference and alignment: identify the feedback first
Basics
What role does the constitution play in Constitutional AI?
Show answer and example
Answer: Explicit natural-language principles guide critique, revision and AI preference judgments. They supply human supervision, not absolute values discovered by the model.
Illustrative example: Use a stated principle to explain a revision instead of an unexplained good/bad score.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why sample positive residual after rejecting a speculative proposal?
Fast Inference from Transformers via Speculative Decoding
Show answer and example
Answer: Accepting by target/proposal ratio already contributes some target mass. Sample normalized positive target-minus-proposal remainder to fill the rest. Resampling target directly double-counts accepted mass. Local token transitions do not measure large kernels.
Illustrative example: When draft overweights one word, residual correction fills undercovered alternatives.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Why not commit later drafts unconditionally after a rejection?
Fast Inference from Transformers via Speculative Decoding
Show answer and example
Answer: Their prefix included the rejected token. Correct the token, discard the invalid suffix, and continue from the new prefix. Local Markov call counts are not T5-XXL latency.
Illustrative example: If word two changes, words three to five cannot assume the old word two.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
Method comparison
Is greedy agreement the same guarantee as exact stochastic speculation?
Fast Inference from Transformers via Speculative Decoding
Show answer and example
Answer: No. Greedy agreement preserves a deterministic argmax path. Exact stochastic sampling needs p/q acceptance and residual correction for the complete distribution.
Illustrative example: Identical highest-probability words can hide different second- and third-choice probabilities.
Sections in the Chinese source: 核心公式, 本地复现. The example illustrates the mechanism.
Basics
Why can a small draft reduce large-target serial calls?
Fast Inference from Transformers via Speculative Decoding
Show answer and example
Answer: Draft proposes blocks and target verifies in a batch, advancing through the accepted prefix. Exact acceptance/rejection correction preserves the target distribution.
Illustrative example: One target call can accept several candidates rather than one per call.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Core mechanism
Why can PAL still answer incorrectly after delegating arithmetic to Python?
Show answer and example
Answer: A deterministic runtime executes the given program, not the intended problem. The model still represents variables, relations, and control flow. Wrong units or discount order yield precisely wrong results. PAL reduces arithmetic errors without eliminating modeling mistakes.
Illustrative example: Example: The task discounts the subtotal before shipping, but code discounts both. Execution is correct for the wrong formula.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Implementation details
Is the local CALL/RETURN intermediate representation arbitrary Python execution?
Show answer and example
Answer: No. A controlled interpreter verifies program construction and execution separation without Codex or paper GSM8K. Narrow actions help check contracts but do not establish open Python synthesis or a general security sandbox.
Illustrative example: Example: Predefined CALL nodes do not cover arbitrary loops, files, or external dependencies in a real runtime.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Method comparison
How do error sources differ in PAL and manual chain-of-thought arithmetic?
Show answer and example
Answer: CoT handles understanding, steps, and arithmetic together. PAL gives exact arithmetic to programs, reducing numerical mistakes but introducing code and interface errors. Inspect intermediates and verify inputs and formulas. Determinism alone cannot predict accuracy or cost.
Illustrative example: Example: An interpreter avoids multiplication mistakes, but treating a tax rate as a monetary amount remains a modeling error.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Basics
Why does PAL generate programs and leave calculation to an interpreter?
Show answer and example
Answer: Word problems require both interpreting relationships and exact computation. Models turn interpretation into runnable code and the interpreter handles arithmetic. This reduces manual calculation burden, assuming the program captures the task. Execution cannot detect omitted conditions.
Illustrative example: Example: Assign people and prices to variables, but forgetting a half-price child means accurate execution of an incorrect total.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Core mechanism
Why must observations return to the next context instead of letting the model continue its own reasoning alone?
Show answer and example
Answer: Actions obtain real external feedback so the next step can revise beliefs and plans. Without observations, the model can treat expectations as actual outcomes and lose the action-reasoning loop. ReAct alternates reasoning, executable actions, and observations, rather than merely mentioning tools in text.
Illustrative example: Example: The model expects a file to exist, but reading it returns not found. Change the search path rather than analyzing imagined contents.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Implementation details
Why distinguish generated tokens from environment tokens during ReAct training or evaluation?
Show answer and example
Answer: Tool observations are external state, not policy actions. Do not count them as generated capability or train policy gradients on them. The local deterministic suite verifies the loop and counts; the separate isolated capability protocol also prevents the policy from reading references. Shared method names do not make the evidence interchangeable.
Illustrative example: Example: A 100-word tool return is not 100 words generated successfully by the policy, and hidden answers must not enter a visible reference field.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Method comparison
How do ReAct and pure chain-of-thought differ in when errors surface?
Show answer and example
Answer: Pure chain-of-thought can keep developing internal assumptions without confronting external facts. ReAct gets observations through action and can revise plans mid-run. Feedback may still be incomplete or tools may fail, so alternation does not guarantee facts; verify tool results and their support for the conclusion.
Illustrative example: Example: Search can reveal that author B, not A, wrote a book. Pure internal reasoning may continue inventing details about A.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Planning and search: what should happen next?
Basics
Why does ReAct put answering and interacting with the outside world in one loop?
Show answer and example
Answer: Internal knowledge may lack current facts and cannot establish whether an action succeeded. ReAct interleaves reasoning with tool or environment calls and uses results in later decisions. This fits tasks mixing information and actions. Observations support correction, but tool failures and misinterpretation can still cause errors.
Illustrative example: Example: Check inventory before deciding whether to order or choose an alternative, rather than guessing stock from general knowledge.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Core mechanism
What do option tuning and adapters change in M6-Rec's multitask setup?
M6-Rec: Generative Pretrained Language Models are Open-Ended Recommender Systems
Show answer and example
Answer: The frozen backbone preserves pretrained capability. Option parameters provide a small task decision interface, and layerwise adapters add compact task-specific transforms. Shared parameter-efficient adaptation does not automatically make inference cheap; edge deployment needs separate compression and serving design. Local small tasks do not demonstrate full giant-M6 transfer.
Illustrative example: Illustration: Use different options and adapters for relevance and content explanation on one frozen model, but both still execute the large backbone.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Implementation details
Why check normalization and temperature for M6-Rec's local frozen-backbone binary option scores?
M6-Rec: Generative Pretrained Language Models are Open-Ended Recommender Systems
Show answer and example
Answer: The source scores normalized representations against trainable prototypes, with temperature controlling the class distribution. Without normalization, norms can change the mechanism. Freeze both backbones and update only options and permitted adapters. BERT-tiny is a reduced task path, not full generative M6.
Illustrative example: Illustration: An unrestricted prototype can win by growing its norm instead of learning direction; normalization and temperature enforce the defined scoring.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Method comparison
What does option-adapter tuning learn beyond option tuning?
M6-Rec: Generative Pretrained Language Models are Open-Ended Recommender Systems
Show answer and example
Answer: Option prototypes mainly change the output decision interface. Adapters add low-rank nonlinear transformations between frozen layers and can adapt intermediate representations. They still use far fewer parameters than full fine-tuning, but the small and variable local trend does not establish stable production gains.
Illustrative example: Illustration: Moving yes/no prototypes changes the ruler; adapters can change the features it measures. The two arms do not have identical parameter budgets.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Semantic representations, user knowledge, and cold start
Basics
Why turn anonymous behavior and candidates into natural language for M6-Rec?
M6-Rec: Generative Pretrained Language Models are Open-Ended Recommender Systems
Show answer and example
Answer: A text interface lets one pretrained model receive different recommendation problems. Options and adapters encode task differences without maintaining a full model per scenario. Retrieval, explanation, and classification still have distinct outputs and decoding costs.
Illustrative example: Illustration: The same model can make a yes/no movie recommendation and explain it; explanation accuracy is not retrieval hit rate.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Core mechanism
Why doesn't routing to deterministic modules guarantee a correct system answer?
Show answer and example
Answer: Calculators and databases are reliable only within their contracts. Routers may choose the wrong module or inputs, and language synthesis can misread outputs. MRKL separates precise capabilities from generation. Local determinism and end-to-end correctness are different guarantees.
Illustrative example: Example: A calculator receives an incorrect tax rate. Its result is precise and reproducible but wrong for the user.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Implementation details
Do local specialist call counts include Jurassic-X or a real knowledge base?
Show answer and example
Answer: No. Deterministic calculator and calendar specialists test routing and counts without Jurassic-X, search, or private databases. Proxy cost is not online latency. Symbolic call counts do not establish open QA or fresh knowledge quality.
Illustrative example: Example: A fixture database record tests scheduling, not real permissions, failures, or data updates.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Method comparison
What do modular experts trade off against one LLM doing everything?
Show answer and example
Answer: Experts can improve control of exact computation or fresh knowledge. One LLM has simpler interfaces but can rely on stale internal guesses. Routing, conversions, and errors add modular costs and depend on expert coverage. A calculator does not confer provable whole-system correctness.
Illustrative example: Example: A calendar computes the date difference; explaining why something was delayed still needs language and evidence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Basics
Why does MRKL combine an LLM with several external experts?
Show answer and example
Answer: Open questions can need language understanding, precise calculation, database access, or specialized knowledge. The LLM routes work and integrates results. Modularity separates capability sources but introduces routing and interface risks. One reliable module does not make the whole system reliable.
Illustrative example: Example: Fetch a balance from a database, calculate the difference, then explain it. Querying the wrong account invalidates even exact arithmetic.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Core mechanism
Why multiply language relevance by affordance to filter plausible but impossible actions?
Show answer and example
Answer: Language scoring measures fit to instructions and history; the value function estimates execution success in the current world. Both need to be high. Language alone can suggest unreachable objects, and feasibility alone can pick irrelevant skills. Coverage and value estimates still limit selection.
Illustrative example: Example: A cup in a locked cabinet is relevant but not directly graspable. Opening the cabinet first can score better.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Implementation details
Do local affordance checks validate real value-function training?
Show answer and example
Answer: No. Deterministic feasibility filters required and distractor skills while preserving two-score step selection. No vision, robot policy, or value network was trained. Check counts show execution paths, not calibrated success predictions or physical task results.
Illustrative example: Example: A rule can block grasping with a full hand without estimating reachability from images or grasp probability.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Method comparison
Why are plan success and execution success different robot metrics?
Show answer and example
Answer: A sound plan can fail through perception, grasp, or control errors. Execution includes physical risk. SayCan reports both; higher planning rates are not completion rates. A deterministic local gate does not cover real control failures.
Illustrative example: Example: Picking up a cup and placing it on a table is a valid plan. A slipping gripper still makes execution fail.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Basics
Why can't a robot directly execute a plausible language-model plan?
Show answer and example
Answer: Language reasoning may not know robot position, skill limits, or physical conditions. SayCan connects task meaning to affordance, selecting related and feasible next steps. Feasibility estimates can err, so report simulated plans separately from real completion.
Illustrative example: Example: Getting a cup is semantically right for a drink request, but the robot may need an executable approach step if it cannot reach the cup.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Core mechanism
Why distinguish reward model, critic, old policy and reference?
Show answer and example
Answer: Reward model scores preferences, critic predicts returns for advantages, old policy records sampling, and reference anchors SFT drift. Shared initialization doesn't merge roles. Candidate updates aren't human ranking, reward-model training or large-scale RLHF.
Illustrative example: A reward model rates a reply, a critic forecasts its return, and reference measures deviation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Implementation details
Does local PPO-RLHF include a learned reward model?
Show answer and example
Answer: No. Deterministic outcome/process axes reward a linear candidate policy. Actor, old policy, critic, clip and KL structure do not imply GPT-3 training or human data.
Illustrative example: A numerical critic can verify baseline updates without reproducing RLHF.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Method comparison
Why distinguish PPO clipping from reference KL?
Show answer and example
Answer: Clipping limits a proximal step relative to old policy; KL anchors drift relative to a reference over longer horizons. They aren't substitutes and don't prevent reward exploitation.
Illustrative example: Repeated small steps can still travel far from the initial policy.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
LLM post-training / Policy optimization: which policy samples and which policy learns?
Basics
What supervision enters each InstructGPT stage?
Show answer and example
Answer: Human demonstrations train SFT; human rankings train a reward model; PPO optimizes reward with reference KL. Models and data differ, so a PPO loss alone is not the pipeline.
Illustrative example: Having demonstrations without rankings doesn't mean a reward model was trained.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Core mechanism
Why rank citation-bearing browsing trajectories by human preference instead of simply requiring links?
Show answer and example
Answer: A URL does not guarantee support or answer quality. Behavior cloning learns browser actions, then preference reward selects better trajectories. Citations are checkable entry points, but selective quoting or unvisited sources are not reliable and must map to claims.
Illustrative example: Example: Two answers cite the same article; one reports the date correctly and one overstates its conclusion. Link counts cannot distinguish them.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Implementation details
Is choosing between two local trajectories by evidence coverage equivalent to live browsing and a reward model?
Show answer and example
Answer: No. Local checks retain browse actions, citation constraints, and candidate selection without live web, GPT-3 training, or a human preference model. Controlled coverage cannot be renamed human win rate, TruthfulQA, or real web fact verification.
Illustrative example: Example: Selecting a preset trace with broader evidence tests rejection sampling, not the full system without real pages and preference scoring.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Method comparison
What roles do behavior cloning and human-preference rejection sampling play?
Show answer and example
Answer: Cloning learns search, browsing, and answering from demonstrations. A preference model ranks and selects among candidate trajectories. Selection does not create facts, and can incur judge bias and sampling cost. Best-of-64 is not a single-call efficiency comparison.
Illustrative example: Example: Choosing one of 64 responses may improve preference scores while paying for multiple browsing and generation runs.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Agents / Tools and collaboration: turning plans into inspectable actions
Basics
Why does WebGPT include browsing and source support in long-form QA rather than rely on memory?
Show answer and example
Answer: Open questions can need current facts and multiple sources. Browsing collects evidence for cited answers, and human feedback rates quality. Retrieved text still needs interpretation. Citation count neither proves support for individual claims nor covers all material facts.
Illustrative example: Example: Comparing two policies requires both primary descriptions. One related link cannot support every figure in the paragraph.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Core mechanism
Without position vectors, how does ALiBi tell how far apart tokens are?
Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation
Show answer and example
Answer: It subtracts a distance-proportional bias from each head's attention logits. Distant tokens need a stronger content match, and slopes vary by head. The rule extrapolates without a fixed-length table, but that does not guarantee arbitrary-length retrieval.
Illustrative example: Equal content scores at distances 2 and 20 favor the closer token; a distant answer can still win with a stronger match.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Implementation details
Can ALiBi's distance penalty replace causal masking?
Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation
Show answer and example
Answer: No. Bias changes preferences among visible positions, while the mask excludes future ones. Broadcast each head's slope and keep causality. Small local models do not establish the paper's large-scale gain.
Illustrative example: Token 3 cannot read token 4 even though their distance is one.
Sections in the Chinese source: 核心公式, 复现边界. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Method comparison
Where do linear bias and position embeddings act?
Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation
Show answer and example
Answer: Position embeddings modify token inputs before Q/K/V projections. ALiBi does not learn position vectors; it modifies per-head matching logits directly. These are distinct information paths.
Illustrative example: The same content vectors can have different logit penalties at different distances.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Basics
Why does ALiBi address training short and testing long?
Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation
Show answer and example
Answer: Position mechanisms fitted only to training lengths can fail at greater distances. ALiBi computes a distance penalty directly in logits. The positional rule extrapolates; long-context accuracy is empirical.
Illustrative example: After training on 512 words, compute distances for word 800 without looking up a learned 800th position.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Core mechanism
Why do absolute Q/K rotations yield relative position in RoPE dot products?
RoFormer: Enhanced Transformer with Rotary Position Embedding
Show answer and example
Answer: The first rotation's transpose cancels the shared absolute phase, leaving displacement n−m. Q/K subspaces rotate, not values. Evaluability at larger positions does not guarantee trained retrieval extrapolation.
Illustrative example: Moving positions 3 and 8 to 13 and 18 preserves distance 5 and their relative phase.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Implementation details
Can RoPE rotate only Q or only K?
RoFormer: Enhanced Transformer with Rotary Position Embedding
Show answer and example
Answer: No. Relative displacement requires both R_m and R_n. Pair channels correctly and continue absolute positions through incremental decode.
Illustrative example: A single input token at position 100 still uses rotation 100, not zero.
Sections in the Chinese source: 核心公式, 复现边界. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Method comparison
How does RoPE differ from adding absolute embeddings?
RoFormer: Enhanced Transformer with Rotary Position Embedding
Show answer and example
Answer: Absolute embeddings modify representations additively. RoPE rotates Q/K so the inner product can express a relative rotation. These are different operations.
Illustrative example: Shifting both segments equally preserves RoPE's relative phase.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Long context: attention, recurrent state, and memory
Basics
Why rotate two-dimensional Q/K subspaces?
RoFormer: Enhanced Transformer with Rotary Position Embedding
Show answer and example
Answer: Position determines rotation angle, so dot products depend on relative phase and content without learning one vector per position.
Illustrative example: Identical content at distances 2 and 20 can produce different scores.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Core mechanism
Why does CLIP use both image-to-text and text-to-image contrastive learning?
Learning Transferable Visual Models From Natural Language Supervision
Show answer and example
Answer: The same batch supplies two classification problems: find each image's paired text and each text's paired image. Both directions shape the shared similarity space. Other in-batch pairs provide negatives, but overall matching scores do not establish fine-grained factual accuracy.
Illustrative example: Cat and dog images pair with their respective descriptions, and both retrieval directions should rank each matching pair first.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Implementation details
Why normalize each vector before computing CLIP's similarity matrix?
Learning Transferable Visual Models From Natural Language Supervision
Show answer and example
Answer: Normalization makes dot products track direction instead of letting vector norms inflate scores; temperature controls sharpness. Matching pairs are on the diagonal, and the two directional losses use the matrix and its transpose. Not every pairing is a positive.
Illustrative example: Doubling an image vector should not make it more correct for every text. With unit normalization only a direction change alters cosine similarity.
Sections in the Chinese source: 核心公式, 本地复现, 复现边界. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Method comparison
How do CLIP's class text embeddings differ from autoregressive label generation?
Learning Transferable Visual Models From Natural Language Supervision
Show answer and example
Answer: CLIP compares similarities among supplied class texts and returns a choice or ranking within the candidates. An autoregressive model predicts a label token sequence that requires generation and parsing. CLIP's descriptions are input prototypes, not generated targets, so this interface does not provide open-ended captioning.
Illustrative example: Given “a photo of a cat” and “a photo of a dog,” CLIP compares their scores without generating “This is a cat.”
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Multimodal models / Cross-modal alignment: representations, connectors, and supervision
Basics
Why can CLIP do zero-shot classification without retraining a classifier head?
Learning Transferable Visual Models From Natural Language Supervision
Show answer and example
Answer: Separate encoders normalize image and class-description vectors and compare them in a shared space. The descriptions are class prototypes and can be replaced to define new labels. This depends on pretraining and does not guarantee arbitrary new concepts or fine-grained relationships are recognized.
Illustrative example: Compare a shoe image with descriptions such as “a shoe” and “a coat,” then choose the higher similarity without training another linear head.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Core mechanism
Why does top-1 Switch still need load balancing?
Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity
Show answer and example
Answer: Top-1 caps experts per token but may concentrate traffic, bottlenecking winners and starving other experts. The auxiliary term combines actual dispatch proportions and mean router probabilities. More parameters do not imply proportional throughput.
Illustrative example: One of eight experts receiving 90% of tokens can overload despite one FFN per token.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Implementation details
What do f_i and mean p_i count in Switch's load term?
Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity
Show answer and example
Answer: f_i is actual dispatch frequency; mean p_i is average router probability. Preserve the chosen expert's gate. Local code does not cover capacity overflow or cross-device all-to-all.
Illustrative example: High dispatch and probability may correlate but are separate statistics.
Sections in the Chinese source: 核心公式, 复现边界. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Method comparison
How does true top-1 dispatch differ from computing all outputs then one-hot masking?
Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity
Show answer and example
Answer: Sparse dispatch executes selected experts only. Dense masking can match outputs while paying every expert's cost. Inspect actual execution.
Illustrative example: Computing four experts and discarding three is not one-expert compute.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Foundation models / Conditional computation: experts, elastic depth and width, and lookup memory
Basics
How does Switch increase parameters without using all experts per token?
Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity
Show answer and example
Answer: Replace the FFN with experts and dispatch each token to one. This grows capacity without computing all expert branches, although routing and communication remain.
Illustrative example: Four experts can be available while one runs per token.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Core mechanism
Why does PLE distinguish shared and private experts beyond MMoE's task-specific gates?
Show answer and example
Answer: MMoE mixes differently but all experts receive multiple tasks' gradients. PLE reserves private capacity alongside shared experts and extracts common and task-specific information layerwise. This changes information and gradient paths at extra capacity and routing cost; verify whether regressions actually decline.
Illustrative example: Illustration: Share category interest while purchase-private experts learn affordability and click-private experts learn headline appeal.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Implementation details
Why restrict each PLE task gate to its private experts and shared experts?
Show answer and example
Answer: This is the CGC separation contract. Accessing another task's private group changes the topology. Verify expert identities, masks, actual gradients, and next-layer channel layout rather than trusting variable names.
Illustrative example: Illustration: Concatenating all experts without a mask can preserve output shape but let click read purchase-private features.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Method comparison
Why isn't PLE just increasing MMoE expert count?
Show answer and example
Answer: More shared experts still accept all tasks' gradients. PLE changes access and gradient topology with private groups and layered fusion, not just parameter count. Match budgets and do not generalize a local two-task comparison to every conflict.
Illustrative example: Illustration: Ten additional shared experts can still transmit click interference to purchases; a purchase-private group blocks that route.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Basics
Why can average task AUC hide PLE's seesaw effect?
Show answer and example
Answer: One task can improve while another regresses and the average obscures the loser. Report each objective under the same joint training. This differs from measurement noise and does not require identical gains across tasks.
Illustrative example: Illustration: Click AUC gains two points and purchase loses one; average AUC rises without both improving.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why does each DCN-V2 cross layer multiply the current transform by the original input x₀?
DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems
Show answer and example
Answer: The original anchor creates traceable higher-order combinations as depth increases instead of relying solely on implicit MLP interactions. Matrix crosses increase expressivity; low-rank factors and expert gating control cost. Compression can reduce capacity and must be tested with the deep path.
Illustrative example: Cross user and category first, then multiply by the original time feature to form a third-order interaction.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Which layers provide x0 and xl in DCN-V2?
DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems
Show answer and example
Answer: x0 is always the cross stack's original input and xl its current state; replacing x0 with xl changes the cross. Map each low-rank expert back to original width before multiplication and normalize gates over experts. Matching residual shapes does not verify the operator.
Illustrative example: Layer two still multiplies the original user and item vectors rather than squaring the previous state, despite matching shapes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
What is the capacity-compute trade-off between DCN-V2 low-rank expert mixing and a full cross matrix?
DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems
Show answer and example
Answer: U and V factor the large matrix through a smaller subspace. Multiple gated experts add input-dependent flexibility but also storage and routing. A single low-rank expert limits representable directions; parameter savings do not imply equal end-to-end latency savings because embeddings and other modules still cost time.
Illustrative example: Project thousands of dimensions to dozens and back, with separate expert crosses. Too little rank can lose rare, important relationships.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why does depth control the order of explicit crosses in DCN-V2?
DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems
Show answer and example
Answer: Each layer elementwise multiplies a transform of current state by the original input, building higher orders while residuals retain lower ones. This gives traceable polynomial interaction paths, unlike generic MLP fitting. Rank, width, and training still constrain effective expressivity.
Illustrative example: A user age–category interaction can gain another original feature at the next layer without losing its lower-order residual.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why use GSU then ESU instead of scoring lifelong history directly?
Search-based User Interest Modeling with Lifelong Sequential Behavior Data for CTR Prediction
Show answer and example
Answer: Most history is irrelevant per candidate. Cheap GSU narrows it, then ESU attends precisely to selected behavior. Fine matching cannot recover evidence omitted by coarse retrieval.
Illustrative example: If category filtering excludes old lens interactions, later camera attention cannot see them.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
How prevent future leakage in SIM top-k history?
Search-based User Interest Modeling with Lifelong Sequential Behavior Data for CTR Prediction
Show answer and example
Answer: GSU searches only pre-request history, separate from the target label. ESU sees selected entries only. MovieLens verifies mechanism, not production or online A/B.
Illustrative example: Do not add the next actual click before searching history.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does candidate-related selection differ from one user vector?
Search-based User Interest Modeling with Lifelong Sequential Behavior Data for CTR Prediction
Show answer and example
Answer: Different candidates retrieve different evidence. A shared user vector compresses once and is easy to cache. Quality and cost still need measurement.
Illustrative example: Shoes and books can activate different historical interests for one user.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why run GSU search before ESU attention?
Search-based User Interest Modeling with Lifelong Sequential Behavior Data for CTR Prediction
Show answer and example
Answer: Lifelong histories are long. Cheap candidate-related retrieval narrows them before fine matching, rather than always taking recent events.
Illustrative example: A camera candidate can retrieve older photography behavior.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
How does BST jointly attending over a candidate and history differ from DIN using a candidate to weight history?
Behavior Sequence Transformer for E-commerce Recommendation in Alibaba
Show answer and example
Answer: DIN mainly learns candidate-to-action activation. BST also enables interactions among past actions and embeds the candidate jointly, allowing context to change an action's meaning. This costs more sequence computation and can limit reuse across candidates; compare matched budgets and history windows.
Illustrative example: A camera click might be incidental, but later lens and tutorial views reinforce photography intent. A different candidate can change this jointly encoded state.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why not cache BST's candidate-conditioned state as a user embedding shared by all candidates?
Behavior Sequence Transformer for E-commerce Recommendation in Alibaba
Show answer and example
Answer: A candidate participating in self-attention can write into hidden history states; substituting it changes downstream inputs. Without an isolation mask, caching is valid only for the specific dependency conditions. The local candidate-aware Transformer versus DIN is not a demonstration of shared user-side caching.
Illustrative example: A state computed for shoes can emphasize shoe history and bias a movie score. A shared cache must stop before candidate information is written.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
What does BST's self-attention add over fixed history pooling, and at what cost?
Behavior Sequence Transformer for E-commerce Recommendation in Alibaba
Show answer and example
Answer: Fixed pooling uses predetermined or simplified action weights and can miss combinations. BST learns pairwise contextual dependencies, with sequence-length-related computation and possible re-encoding for each candidate. Flexibility alone is not a win; compare matching data, budgets, and candidate protocols.
Illustrative example: A camera view followed by a lens view can reflect a different purchase stage than a camera view alone. Attention can combine them but costs more with thousands of events.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why include BST's candidate token in history encoding rather than concatenating it only in the final MLP?
Behavior Sequence Transformer for E-commerce Recommendation in Alibaba
Show answer and example
Answer: Attention with the candidate lets history representations adapt early and learns both behavior-to-behavior and target-to-history relations. Late concatenation only interacts with an already compressed representation. The result is candidate-conditioned, not universal.
Illustrative example: Hiking shoes can amplify a related camping and hiking history, while a camera should refocus the encoding on photography.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
How does DIEN multiplying target relevance into a GRU update gate differ from final history pooling?
Deep Interest Evolution Network for Click-Through Rate Prediction
Show answer and example
Answer: Auxiliary prediction learns per-step interests, and candidate attention modulates each state update, letting relevant events revise memory while suppressing irrelevant changes. Late pooling cannot undo earlier overwrites. More recurrent computation need not help short histories.
Illustrative example: For a lens recommendation, a random food-ad click should weakly update a strong photography state rather than overwrite it before final pooling.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
How should DIEN auxiliary next-event supervision avoid confusing training labels with test futures?
Deep Interest Evolution Network for Click-Through Rate Prediction
Show answer and example
Answer: Use next events as training labels while extraction inputs stop at current time and negatives follow training rules. Do not feed a held-out event into the GRU at evaluation. Target gating must update recurrent state step by step, not merely weight the final vector.
Illustrative example: Predict event three from position two during training; feeding event three when evaluating that prediction leaks the answer.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
When does candidate information enter DIEN compared with DIN target attention?
Deep Interest Evolution Network for Click-Through Rate Prediction
Show answer and example
Answer: DIN weights history at aggregation, whereas DIEN multiplies relevance into each recurrent update gate, controlling retained and new state throughout evolution. Extra computation and training are not guaranteed improvements; the local gain did not reproduce.
Illustrative example: A lens view can revise recurrent state immediately for a camera candidate; late weighting only reads what survives existing compression.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why separate interest extraction and target-related evolution in DIEN?
Deep Interest Evolution Network for Click-Through Rate Prediction
Show answer and example
Answer: The first stage extracts predictive interest states from behavior with auxiliary supervision. The second uses candidate relevance to control their update strength for the target. This avoids equating raw clicks with interests and remains sensitive to auxiliary negatives and noisy histories.
Illustrative example: Camera and lens views establish photography interest, which matters more for a travel camera than for a novel.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
How does SASRec balance recent and distant interests without reading future?
Self-Attentive Sequential Recommendation
Show answer and example
Answer: Content attention accesses several earlier behaviors beyond fixed-order Markov neighbors, without RNN recurrence. Position representations preserve order and causal masks restrict reads. Next item is supervision only, and attention can still read irrelevant history.
Illustrative example: Recall old sci-fi interest despite recent comedy, while position three cannot read item four.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
How preserve SASRec embeddings and negative objective?
Self-Attentive Sequential Recommendation
Show answer and example
Answer: Tie input item embeddings with output weights and use per-position positive and sampled-negative pairwise BCE, not catalog CE. Mask padding and future positions.
Illustrative example: Never sample the true positive as its own negative.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does causal next-item prediction differ from masked item recovery?
Self-Attentive Sequential Recommendation
Show answer and example
Answer: Causal training uses only history; masked recovery can use both neighbors. Right context legal for recovery is future leakage in next-item prediction.
Illustrative example: Recovering A,[MASK],C may read C; predicting B after only A may not.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
How does SASRec shift inputs and next-item targets?
Self-Attentive Sequential Recommendation
Show answer and example
Answer: Position i reads history through i and supervises the following item, without exposing future events. Exclude padding; each sequence yields several prefix tasks.
Illustrative example: Use A→B and A,B→C; the representation predicting B cannot read C.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Core mechanism
Why can shared experts with task-specific gates in MMoE reduce shared-bottom negative transfer?
Modeling Task Relationships in Multi-task Learning with Multi-gate Mixture-of-Experts
Show answer and example
Answer: A shared bottom forces one intermediate representation and can mix conflicting gradients. MMoE lets each task conditionally combine jointly trained experts, sharing useful parts while selecting different mixtures. It enables soft specialization but does not guarantee interpretable experts or eliminate conflict.
Illustrative example: Illustration: Click and purchase can both use category knowledge while one emphasizes novelty and the other price.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Implementation details
Why normalize MMoE gates across experts, and how many sets of weights do four experts and two tasks need?
Modeling Task Relationships in Multi-task Learning with Multi-gate Mixture-of-Experts
Show answer and example
Answer: Each task has a distinct four-weight distribution per sample and combines same-space expert outputs. Normalizing across tasks makes them compete for one budget and changes the algorithm. The four-expert, two-head local control is shared-bottom.
Illustrative example: Illustration: Click and purchase each need four weights summing to one; normalizing all eight together couples them incorrectly.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Method comparison
What trade-off does MMoE offer between a shared bottom and independent task networks?
Modeling Task Relationships in Multi-task Learning with Multi-gate Mixture-of-Experts
Show answer and example
Answer: A shared bottom returns one representation, independent networks share no parameters, and MMoE offers several common experts combined per task. Extra experts add flexibility, computation, and collapse risk. Task relationships and data determine whether the compromise helps.
Illustrative example: Illustration: Two tasks can share interest features but differ elsewhere. Unrelated tasks may suffer from sharing, while scarce data can make independent models unstable.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Basics
Does sharing experts but keeping private gates make MMoE tasks independent?
Modeling Task Relationships in Multi-task Learning with Multi-gate Mixture-of-Experts
Show answer and example
Answer: No. Shared expert gradients still transmit effects between tasks. Private gates change expert weights and task heads produce separate outputs. This lies between full sharing and full separation. Check actual routes and task results for specialization.
Illustrative example: Illustration: Click and purchase use the same price expert at different rates; click training still changes the expert that purchase reads.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Core mechanism
Why train CTR and CTCVR in ESMM rather than treating nonclicks as CVR negatives?
Entire Space Multi-Task Model: An Effective Approach for Estimating Post-Click Conversion Rate
Show answer and example
Answer: CVR is conditional on clicking; no click does not reveal post-click purchase. CTR × CVR represents click-and-convert probability, enabling full-impression CTR and joint supervision to address sparsity and selection. The probability chain does not solve attribution, delayed feedback, or causal identification automatically.
Illustrative example: 10% CTR and 20% CVR give 2% joint rate. A nonclick labels click and joint events zero but cannot label hypothetical post-click conversion.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Implementation details
Which ESMM losses use nonclicks, and which head should not be directly supervised by them?
Entire Space Multi-Task Model: An Effective Approach for Estimating Post-Click Conversion Rate
Show answer and example
Answer: Full-impression click labels supervise CTR; full-impression CTCVR labels supervise CTR × CVR. Do not add direct CVR negatives for nonclicks. The local rating-threshold nesting validates wiring, not real ad conversions.
Illustrative example: A nonclick can lower joint probability via low CTR. Forcing CVR to zero adds an unsupported conditional label.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Method comparison
What impression information does ESMM add beyond CVR learned only in clicked space?
Entire Space Multi-Task Model: An Effective Approach for Estimating Post-Click Conversion Rate
Show answer and example
Answer: Clicked examples reflect user selection and the prior policy. Full-space CTR and CTCVR let nonclicks constrain joint outcomes, and shared features help sparse conversion. Neither AUC nor full-space supervision resolves delayed feedback or all confounding.
Illustrative example: Skipped expensive phones supply impression information, not direct counterfactual conversions if clicked.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Learning signals, multitask learning, and feedback bias
Basics
Why is CTCVR defined on all ESMM impressions but conditional CVR is not?
Entire Space Multi-Task Model: An Effective Approach for Estimating Post-Click Conversion Rate
Show answer and example
Answer: Nonclicks are known negatives for click-and-convert, but whether they would convert if clicked is unobserved. The CTR–CVR product enables full-space joint supervision only under nested click-to-conversion semantics.
Illustrative example: An unclicked ad did not produce a post-click purchase, but this says nothing definitive about purchase under a click.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why can DIN produce different interest vectors for the same user?
Deep Interest Network for Click-Through Rate Prediction
Show answer and example
Answer: Which historical actions matter depends on the candidate. DIN learns activations from each candidate-action feature combination and pools accordingly. Greater expressivity prevents blanket reuse of the resulting state across candidates; compare against matched-budget pooling.
Illustrative example: Running socks emphasize running-shoe history, while baking tools emphasize oven activity in the same user history.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Over which axis should DIN normalize weights, and how should it handle padding?
Deep Interest Network for Click-Through Rate Prediction
Show answer and example
Answer: Normalize over valid history positions for one user and candidate, masking padding first. Do not normalize across candidates or batch examples. Keep Dice training statistics and serving state consistent without adjusting from test labels.
Illustrative example: With three actions and five pads, distribute weights only over the three valid positions.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does DIN local activation differ from dot-product attention?
Deep Interest Network for Click-Through Rate Prediction
Show answer and example
Answer: It takes the candidate, action, difference, and elementwise product to learn richer matching. Dice adapts its boundary to feature distributions. This does not imply causality or guaranteed gain; the local matched comparison showed no improvement over mean pooling.
Illustrative example: Near-equal dot products can hide price differences and shared categories that the difference and product features highlight.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why is DIN local interest candidate-conditioned rather than a single short-term interest?
Deep Interest Network for Click-Through Rate Prediction
Show answer and example
Answer: Local means the relevant subset of all history for the current target. A user can have several interests; candidate changes alter activations and aggregation without requiring one label. Attention cannot create missing evidence, and low activation does not imply permanent disinterest.
Illustrative example: The same user can like science fiction and cooking, activating different histories for a movie and cookware.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why do DeepFM's FM and DNN share field embeddings?
DeepFM: A Factorization-Machine based Neural Network for CTR Prediction
Show answer and example
Answer: FM explicitly models first- and second-order terms and the DNN learns higher-order nonlinear interactions. Shared field semantics reduce duplicated parameters and manual crosses and let both paths train the embeddings. Their coupled gradients need not be conflict-free.
Illustrative example: FM learns occupation–category pairs, while DNN combines occupation, category, and price; both update the same field vectors.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why sum only i<j in DeepFM's second-order term?
DeepFM: A Factorization-Machine based Neural Network for CTR Prediction
Show answer and example
Answer: FM models distinct feature pairs. Self-products add squares and counting both orders changes scale. Efficient computation uses half the difference between the square of the sum and the sum of squares. The FM and DNN must share parameters, not copies with the same initial values.
Illustrative example: Count age–category once, not age–age. Copies of vectors would diverge once the two paths train.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How do DeepFM's engineering assumptions differ from hand-crossed Wide & Deep?
DeepFM: A Factorization-Machine based Neural Network for CTR Prediction
Show answer and example
Answer: FM learns pairwise relations from vector inner products and shares fields with the deep path, reducing manual crosses. Explicit crosses offer exact rules but cost coverage and maintenance. Embedding limits remain, as do field design and governance.
Illustrative example: FM can learn city–category effects without one column per pair, but an illegal business combination still needs an explicit constraint.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
What do DeepFM's first-order, second-order, and DNN outputs contribute?
DeepFM: A Factorization-Machine based Neural Network for CTR Prediction
Show answer and example
Answer: First-order terms give individual effects, FM vector products capture pairs, and the DNN learns richer nonlinear interactions. They meet in one click logit with shared embeddings; orders can overlap and probabilities should not be added.
Illustrative example: A city effect, a city–category pair, and a more complex price/time condition all contribute to the same logit.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Core mechanism
Why train next watch as classification but serve nearest neighbors?
Deep Neural Networks for YouTube Recommendations
Show answer and example
Answer: A nonlinear user tower and item dot products define training logits. The same dot-product space supports indexing candidates, avoiding a complex per-video forward. Recall scores are not final ranking utility.
Illustrative example: Compute one user vector, retrieve from a million-item index, and then apply fine ranking.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Implementation details
How align next-watch softmax training and serving retrieval?
Deep Neural Networks for YouTube Recommendations
Show answer and example
Answer: Use the same score space and item-vector version with time-causal history. Do not mix sampled training probabilities and full-catalog metrics. The public implementation lacks the original inventory and stack.
Illustrative example: Refresh the item index after user-space changes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Method comparison
What does a deep user tower add over matrix factorization?
Deep Neural Networks for YouTube Recommendations
Show answer and example
Answer: It dynamically combines history/context instead of looking up fixed user/item factors. One vector can still compress away distinct interests.
Illustrative example: Recent preference shifts can enter the tower without retraining a user ID vector.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / Candidate generation and search matching
Basics
Why use a deep user tower for YouTube candidate generation?
Deep Neural Networks for YouTube Recommendations
Show answer and example
Answer: Combine heterogeneous history and context embeddings into a user vector, train next-watch classification, and retrieve neighbors. Fine ranking is separate.
Illustrative example: Narrow millions of videos before applying richer ranking features.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why train crosses and deep features jointly rather than average independent scores?
Wide & Deep Learning for Recommender Systems
Show answer and example
Answer: Both contribute one logit and share objective gradients, allowing complementary learning. Handcrafted crosses have limited coverage and deep generalization can be wrong.
Illustrative example: Wide memorizes a frequent user–category cross while deep transfers to related new combinations.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Does local Wide & Deep use production Google Play crosses?
Wide & Deep Learning for Recommender Systems
Show answer and example
Answer: No. MovieLens replaces private logs and distributed training/serving. Add logits before sigmoid and train jointly, rather than averaging post-sigmoid probabilities.
Illustrative example: sigmoid(a+b) generally differs from averaging sigmoid(a) and sigmoid(b).
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How can wide memorization and deep generalization each fail?
Wide & Deep Learning for Recommender Systems
Show answer and example
Answer: Memorization misses unseen crosses, while similarity can overgeneralize to incompatible combinations. Capacity should match sparsity and feature engineering.
Illustrative example: Semantic similarity does not establish accessory compatibility.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why keep linear and deep paths together?
Wide & Deep Learning for Recommender Systems
Show answer and example
Answer: Wide memorizes sparse observed crosses; deep embeddings generalize to unseen combinations. One jointly trained logit combines them, rather than alternating independent services.
Illustrative example: Memorize frequent combinations and generalize related new ones.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.