Generation paths: draft verification, parallel denoising, and few-step refinement¶
简体中文 · English
Foundation models / 60 questions · 15 papers
Try answering each question before opening the explanation. The bank grows as papers and questions are added.
Read this topic's comparisons, analysis, and cross-paper Q&A
60 questions · 15 papers
Basics
Why choose drafts per request?
ASPIRE: Asynchronous Batch Self-Speculative Decoding
Show answer and example
Answer: Online acceptance and remaining conditions vary. Use current batch cost and per-request statistics, with periodic refresh for state correction.
Illustrative example: A frequent long-draft accepter need not match a frequent rejecter's length.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Core mechanism
Why let ASPIRE choose draft lengths independently within a batch?
ASPIRE: Asynchronous Batch Self-Speculative Decoding
Show answer and example
Answer: Requests differ in acceptance and remaining state. One length can waste checks or lose gains. Use batch costs and periodic full-attention refresh for self-speculation. Local tests do not modify a full engine.
Illustrative example: Give different lengths to frequently accepted and first-token-rejected requests, while accounting for shared batch cost.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
Is per-request choice the same as ignoring batch and acting greedily?
ASPIRE: Asynchronous Batch Self-Speculative Decoding
Show answer and example
Answer: No. Decisions still use shared batch costs; ignoring resource and synchronization effects changes the scheduler.
Illustrative example: Longer drafts can benefit one request but increase group verification cost.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Implementation details
Does the local ASPIRE scheduler and refresh gate constitute a vLLM integration?
ASPIRE: Asynchronous Batch Self-Speculative Decoding
Show answer and example
Answer: No. Verify real scheduling, cache handoff, refresh execution, and time separately. A reference schedule is not deployment throughput.
Illustrative example: A log showing draft length eight does not prove asynchronous service execution or correct KV transfer.
Sections in the Chinese source: 复现边界. The example illustrates the mechanism.
Basics
What changes across Data-Space Iteration refinements?
Few-Step Generation via Data-Space Iteration
Show answer and example
Answer: A shared network updates data predictions from fixed noise. Iteration counts refinement, not the separate DMD noising time τ.
Illustrative example: Refinement three is another prediction update, not diffusion time three.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why refine with the same noise rather than interpreting iteration index as flow time?
Few-Step Generation via Data-Space Iteration
Show answer and example
Answer: The shared generator repeatedly improves one data-space prediction from fixed original noise. DMD uses frozen real teacher and fake distribution model. Earlier refinements are detached and only the current one trains; it is not real-image regression. Local tiny runs do not replace paper metrics.
Illustrative example: Refine outline then detail from the same noise; changing noise or time semantics changes the mechanism.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
Is direct real-image regression equivalent to DMD?
Few-Step Generation via Data-Space Iteration
Show answer and example
Answer: No. DMD uses a frozen real teacher and a fake model fitted to generated data to supply a distribution-matching direction. Per-step paired image regression replaces it.
Illustrative example: A teacher-minus-fake direction on a generated sample is not assigning a fixed real image to noise.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
How isolate prefix, fake model, and generator gradients?
Few-Step Generation via Data-Space Iteration
Show answer and example
Answer: Roll out prefix without a graph and train the final refinement. Train fake on detached samples; freeze teacher/fake gradients during generator updates and detach the direction.
Illustrative example: Updating step four does not backpropagate through the first three steps or teacher.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Basics
Why can individually likely draft positions form a poor block?
DBLast: Dependent Block Drafting for Stochastic Speculative Decoding
Show answer and example
Answer: Independent proposals ignore joint dependencies. DBLast conditions on a shared latent, inducing correlation without abandoning parallelism.
Illustrative example: Common colors and objects can still combine incoherently if chosen independently.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
How does DBLast's shared latent address independent parallel draft positions?
DBLast: Dependent Block Drafting for Stochastic Speculative Decoding
Show answer and example
Answer: Given one latent, positions can sample in parallel; marginalizing it yields dependence across positions. Optimize expected verification length, not isolated hits. Target verification still matters and local mechanism is not service speed.
Illustrative example: One latent can coordinate singular/plural choices instead of independently picking inconsistent words.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
How does a latent mixture differ from independent unigram proposals?
DBLast: Dependent Block Drafting for Stochastic Speculative Decoding
Show answer and example
Answer: Independent proposals multiply marginals. DBLast multiplies conditional probabilities under the same z and integrates z, so identical marginals can have different joints.
Illustrative example: Both positions can be half A/B, with only AA and BB or with all four pairs.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Implementation details
Why check exact speculative acceptance instead of position hit rate?
DBLast: Dependent Block Drafting for Stochastic Speculative Decoding
Show answer and example
Answer: Benefit depends on accepted contiguous prefix and joint quality. Isolated correct positions need not pass as a block. Local Markov target and rank-four proposal are mechanism tests, not real throughput.
Illustrative example: Correct first and third tokens do not help the third if the second is rejected.
Sections in the Chinese source: 背景与主要改动, 本地复现. The example illustrates the mechanism.
Basics
Why reveal some tokens as clean one-hot vectors early?
On the Design Space of Continuous Diffusion Language Models
Show answer and example
Answer: High-dimensional Gaussian noise can swamp a norm-one signal. Revealed anchors remain fixed, while loss covers unrevealed positions.
Illustrative example: Confirmed words stay clean as other positions denoise.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why are DoubleCat two-level interpolation and SemiCat permanent reveal distinct?
On the Design Space of Continuous Diffusion Language Models
Show answer and example
Answer: Interpolation shifts training toward interior simplex states to reduce endpoint mismatch. Permanent reveal leaves chosen tokens clean and fixed, supervises unrevealed positions, and adjusts hazard to remaining time. Neither adds default self-conditioning.
Illustrative example: Keeping a confirmed subject clean differs from training on interior probability patterns.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
Does two-level interpolation require an extra denoiser call for self-conditioning?
On the Design Space of Continuous Diffusion Language Models
Show answer and example
Answer: No. Interpolation constructs inputs without another denoiser. SemiCat also needs no default extra call. Optional bootstrap self-conditioning is a separately costed ablation.
Illustrative example: Do not hide a second forward under a nominal one-call training budget.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Implementation details
How test final hazard one together with permanent carryover?
On the Design Space of Continuous Diffusion Language Models
Show answer and example
Answer: At the last step reveal all remaining positions. Preserve already revealed inputs and outputs at every step. Use untempered probabilities for continuous Euler and temperature only for categorical commits.
Illustrative example: A token revealed at step two cannot change at step three.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Basics
Why convert a hybrid backbone into a bidirectional diffusion LM?
dQwen3.5: Adapting Hybrid Language Models into Bidirectional Diffusion Language Models
Show answer and example
Answer: Masked denoising uses visible context on both sides rather than causal next-token prediction. Attention and recurrent paths, and the objective, all change.
Illustrative example: A masked middle word can use left and right words.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Core mechanism
Why does bidirectional dQwen3.5 need more than removing an attention mask?
dQwen3.5: Adapting Hybrid Language Models into Bidirectional Diffusion Language Models
Show answer and example
Answer: The hybrid backbone also has recurrence. Both state propagation and attention must support bidirectional context for masked denoising. Running a propagation kernel does not establish a fully adapted checkpoint or loss curve.
Illustrative example: A sentence-start blank can need the end; changing only attention leaves recurrence one-way.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
Method comparison
How does masked bidirectional visibility differ from autoregressive training?
dQwen3.5: Adapting Hybrid Language Models into Bidirectional Diffusion Language Models
Show answer and example
Answer: AR is prefix-causal; denoising can use visible left and right context. Adapt the hybrid state path and do not assume causal cache equivalence.
Illustrative example: A known word after a blank is legal evidence for denoising but not next-token AR.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Implementation details
Why test both attention and recurrence for bidirectional conversion?
dQwen3.5: Adapting Hybrid Language Models into Bidirectional Diffusion Language Models
Show answer and example
Answer: Removing the attention mask can leave recurrence one-way. Local CUDA only confirms component execution, not a 9B checkpoint or full denoising curves.
Illustrative example: Right context reaching one branch does not make the other bidirectional.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
Basics
Why explore frequently in early layers but verify less often at the end?
ECHO: Early-layer Collaborative Hierarchical Orchestration with Bonus Logits in Speculative Decoding
Show answer and example
Answer: Early proposals are cheap; full depth verifies reliably. Dual loops and bonus logits expand candidates while aiming to reduce waits and preserve corrected sampling.
Illustrative example: Let the light path expand promising branches for full-depth checking.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Core mechanism
Why not use ECHO bonus logits as the final sampling distribution?
ECHO: Early-layer Collaborative Hierarchical Orchestration with Bonus Logits in Speculative Decoding
Show answer and example
Answer: Frequent cheap early proposals and less frequent authoritative late checks cooperatively expand the candidate tree. Exploration may differ from the target and requires exact rejection correction. Local candidate/tensor checks do not measure throughput.
Illustrative example: A bonus branch may enter verification without being accepted automatically.
Sections in the Chinese source: 背景与主要改动, 核心公式与实现对应. The example illustrates the mechanism.
Method comparison
Why separate exploration scores from target sampling?
ECHO: Early-layer Collaborative Hierarchical Orchestration with Bonus Logits in Speculative Decoding
Show answer and example
Answer: Bonus logits serve proposals and expansion. Exact rejection correction maintains the target distribution. Sampling bonuses directly changes behavior.
Illustrative example: Trying a favored candidate first does not entitle it to higher final probability.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Implementation details
How does candidate merge validation differ from CUDA throughput evidence?
ECHO: Early-layer Collaborative Hierarchical Orchestration with Bonus Logits in Speculative Decoding
Show answer and example
Answer: Reference verifies merge and lossless correction; A100 receipt verifies the tensor path. Real speed needs early/late scheduling, caching, and measured time.
Illustrative example: Distribution parity does not prove ideal GPU overlap.
Sections in the Chinese source: 本地复现, 复现边界. The example illustrates the mechanism.
Basics
Why add a likelihood shortcut besides noisy embeddings?
Designing Transformers for Continuous Diffusion Language Models
Show answer and example
Answer: Codebook-to-noisy-state relations supplement token posterior with noise-conditioned backbone and bounded calibration. This is more than a normal head and needs correct gradients and self-conditioning.
Illustrative example: At low noise, nearby codebook vectors help identify the underlying token.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why detach only the explicit codebook shortcut, not the entire noisy input?
Designing Transformers for Continuous Diffusion Language Models
Show answer and example
Answer: Both paths share a codebook. Detaching shortcut gradient still lets the codebook learn through noisy input; detaching everything breaks it. Training uses entropy-gated self-conditioning while inference uses unweighted posterior mean.
Illustrative example: Freeze one shortcut multiplication, not the same table's input construction path.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
Should training entropy gating also weight inference posterior means?
Designing Transformers for Continuous Diffusion Language Models
Show answer and example
Answer: No. Training bootstrap confidence-attenuates and detaches feedback. Inference uses unweighted pC and no extra bootstrap. Training heuristics are not posterior guarantees.
Illustrative example: Do not discount inference probability mass a second time because training suppressed low-confidence feedback.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Implementation details
Why initialize calibration aψ to one and constrain it to [0.5,2]?
Designing Transformers for Continuous Diffusion Language Models
Show answer and example
Answer: One preserves initial shortcut scale. Bounded exponential calibration avoids extreme amplification or suppression, with 5000 warmup steps. Verify ranges and gradient paths.
Illustrative example: Large calibration outputs must not make the shortcut dominate the vocabulary head.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Basics
Why is self-speculation natural for shared-block loops?
LoopSpec: Pipelined Self-Speculative Decoding for Looped Transformers
Show answer and example
Answer: Intermediate depth provides draft predictions and deeper loops verify. This reuses model weights; pipelined residual proposals reduce intended waiting.
Illustrative example: Draft with fewer loops, then verify with more.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Core mechanism
Why can LoopSpec use intermediate loop predictions without an independent drafter?
LoopSpec: Pipelined Self-Speculative Decoding for Looped Transformers
Show answer and example
Answer: Shallower loop states give cheap predictions and deeper computation verifies them. Pipelined residual proposals aim to overlap waiting. Actual acceptance and scheduling determine gains; a reference scheduler does not measure GPU throughput.
Illustrative example: Reusing shallow drafts saves a separate model but frequent rejection can erase benefits.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
Method comparison
What does LoopSpec reuse compared with an independent drafter?
LoopSpec: Pipelined Self-Speculative Decoding for Looped Transformers
Show answer and example
Answer: It reuses intermediate predictions and shared loop structure, rather than a second parameter set and path. Speed still depends on proposal quality, verification, and scheduling.
Illustrative example: Removing separate drafter weights does not ensure candidates are accepted.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Implementation details
Does a depth scheduler establish real GPU pipelining?
LoopSpec: Pipelined Self-Speculative Decoding for Looped Transformers
Show answer and example
Answer: No. It checks proposal/depth transitions, not actual overlap, cache handoff, or verification execution. Planned concurrency is not measured throughput.
Illustrative example: A log claiming overlap cannot show synchronization did not serialize the GPU.
Sections in the Chinese source: 本地复现, 复现边界. The example illustrates the mechanism.
Basics
Why add multiple future-token heads to one hidden state?
Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads
Show answer and example
Answer: Different offsets provide multiple proposals at once. Tree verification can reduce sequential calls without removing backbone verification.
Illustrative example: Propose t+1 and t+2 from h_t before checking accepted combinations.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why verify a candidate tree rather than concatenate each Medusa head's top token?
Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads
Show answer and example
Answer: Heads predict different offsets from the same state and may not form a valid autoregressive chain. A tree exposes combinations for backbone validation. Independent losses do not prove correctness or sampling guarantees, and local work has no GPU tree kernel.
Illustrative example: Separately chosen nouns and verbs need not form a coherent phrase.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
Why don't fewer backbone calls imply the same percentage latency reduction?
Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads
Show answer and example
Answer: Extra heads, tree construction and validation, batch shape, kernels, and acceptance all matter. Mechanism call counts are not measured GPU speed.
Illustrative example: Halving calls may double the work per verification.
Sections in the Chinese source: 本地复现, 复现边界. The example illustrates the mechanism.
Implementation details
How align Medusa's k-th head targets?
Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads
Show answer and example
Answer: Predict x_{t+k} from h_t, matching slices and valid endpoints. One common next-token target would train duplicate offsets.
Illustrative example: Head two at h_10 predicts x_12, not x_11.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Basics
Why train on the model's own progressive-unmasking trajectory?
On Trajectory-Aware Training for Masked Diffusion Language Models
Show answer and example
Answer: Independent noise may miss states reached by actual multistep sampling. Carry through self-generated unrolls exposes dependent decisions.
Illustrative example: Revealed words determine the next input rather than independently recreating noise.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Core mechanism
Why propagate gradients across several PUMBA carry steps?
On Trajectory-Aware Training for Masked Diffusion Language Models
Show answer and example
Answer: Carry affects future denoising, so isolated loss may not learn useful carry. Unroll on-policy reveals and backpropagate within fixed windows, truncating between windows for cost. Local work is not full diffusion LM training.
Illustrative example: Later fill-in errors should shape how earlier carry represents entity constraints.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
How does trajectory-aware carry training differ from independent steps?
On Trajectory-Aware Training for Masked Diffusion Language Models
Show answer and example
Answer: BPTT lets later loss affect earlier carry. Independent examples cut those dependencies, and summing their CEs does not restore them.
Illustrative example: Step-two loss should influence step-one carry, not only local step-two computation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Should PUMBA detach carry every step or at window boundaries?
On Trajectory-Aware Training for Masked Diffusion Language Models
Show answer and example
Answer: Keep gradients through the fixed window and detach between windows. Per-step detachment removes the mechanism; fully untruncated training changes memory and protocol. Local reference did not train LLaDA-8B.
Illustrative example: At window length three, step three reaches two earlier steps in the same window only.
Sections in the Chinese source: 核心公式, 复现边界. The example illustrates the mechanism.
Basics
Why build a set-level alignment target before training a fast retriever?
Efficient, Property-Aligned Fan-Out Retrieval via RL-Compiled Diffusion
Show answer and example
Answer: Item scores alone may not guarantee whole-set properties. Compile a composite set target into diffusion fan-out to seek fewer online steps.
Illustrative example: Relevant outputs should cover different evidence rather than repeating one top item.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Core mechanism
Why align retrieval sets before compiling into a diffusion fan-out retriever?
Efficient, Property-Aligned Fan-Out Retrieval via RL-Compiled Diffusion
Show answer and example
Answer: Set value includes coverage and diversity beyond item relevance. Define trade-offs with a composite set reward, then train fast diffusion retrieval to approximate it. Reward alignment may be imperfect; local work only implements the set objective.
Illustrative example: Three individually relevant duplicates can be worse than a set covering separate facts.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
Method comparison
How does set alignment differ from independent item ranking?
Efficient, Property-Aligned Fan-Out Retrieval via RL-Compiled Diffusion
Show answer and example
Answer: Sets account for relations and redundancy. Summing independent scores can miss coverage. The compiled retriever requires separate training and evaluation.
Illustrative example: Three high-relevance duplicates can have low diversity.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
Implementation details
Does the local R4T component train the 53.9M diffusion retriever?
Efficient, Property-Aligned Fan-Out Retrieval via RL-Compiled Diffusion
Show answer and example
Answer: No. It computes the alignment/diversity objective, without training the full generator. Scoring candidates is not generating fan-out results.
Illustrative example: A function ranking supplied sets cannot necessarily retrieve a set from a query.
Sections in the Chinese source: 本地复现, 复现边界. The example illustrates the mechanism.
Basics
Why use a fixed number of register tokens across diffusion chunks?
Register Tokens for Bounded-State Reasoning in Diffusion Language Models
Show answer and example
Answer: Hidden states pass some earlier information after text is cleared, while bounding capacity. This is also an information bottleneck.
Illustrative example: The next chunk gets several continuous vectors rather than all previous words.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Core mechanism
How can fixed registers support cross-chunk reasoning after text removal, and what is lost?
Register Tokens for Bounded-State Reasoning in Diffusion Language Models
Show answer and example
Answer: Continuous hidden registers carry compressed previous-chunk information at fixed size. They do not preserve original text or guarantee fine detail. Train and evaluate retention; no complete local diffusion checkpoint.
Illustrative example: A relationship can survive as state while an omitted name cannot be guaranteed recoverable.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
Method comparison
How does register carry differ from keeping all text or KV?
Register Tokens for Bounded-State Reasoning in Diffusion Language Models
Show answer and example
Answer: Carry is a fixed-size continuous summary. Text/KV grows and permits finer position access. Fixed registers are not lossless compression.
Illustrative example: Two hundred pages still compress into eight vectors without expanding capacity.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Implementation details
How verify old text is not retained through an undeclared bypass?
Register Tokens for Bounded-State Reasoning in Diffusion Language Models
Show answer and example
Answer: Check that the next chunk receives only the agreed registers, clear old text and undeclared caches, and verify state updates. Local work omits LLaDA/Dream and diffu-GRPO.
Illustrative example: Fixed register count is insufficient if the old text is secretly concatenated anyway.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
Basics
Why can a small draft reduce large-target serial calls?
Fast Inference from Transformers via Speculative Decoding
Show answer and example
Answer: Draft proposes blocks and target verifies in a batch, advancing through the accepted prefix. Exact acceptance/rejection correction preserves the target distribution.
Illustrative example: One target call can accept several candidates rather than one per call.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why sample positive residual after rejecting a speculative proposal?
Fast Inference from Transformers via Speculative Decoding
Show answer and example
Answer: Accepting by target/proposal ratio already contributes some target mass. Sample normalized positive target-minus-proposal remainder to fill the rest. Resampling target directly double-counts accepted mass. Local token transitions do not measure large kernels.
Illustrative example: When draft overweights one word, residual correction fills undercovered alternatives.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
Is greedy agreement the same guarantee as exact stochastic speculation?
Fast Inference from Transformers via Speculative Decoding
Show answer and example
Answer: No. Greedy agreement preserves a deterministic argmax path. Exact stochastic sampling needs p/q acceptance and residual correction for the complete distribution.
Illustrative example: Identical highest-probability words can hide different second- and third-choice probabilities.
Sections in the Chinese source: 核心公式, 本地复现. The example illustrates the mechanism.
Implementation details
Why not commit later drafts unconditionally after a rejection?
Fast Inference from Transformers via Speculative Decoding
Show answer and example
Answer: Their prefix included the rejected token. Correct the token, discard the invalid suffix, and continue from the new prefix. Local Markov call counts are not T5-XXL latency.
Illustrative example: If word two changes, words three to five cannot assume the old word two.
Sections in the Chinese source: 背景与主要改动, 复现边界. The example illustrates the mechanism.
Basics
Why needn't the expensive anchor run at every step?
Time-Anchored Diffusion Language Models: Latent-Space Caching for Fast Generation
Show answer and example
Answer: Neighboring reverse steps can reuse part of the state. Refresh periodically and use cheap state-conditioned corrections in between before smaller denoising.
Illustrative example: Several steps share one anchor but produce different corrections.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why gate corrections to TADM cached anchors based on current diffusion state?
Time-Anchored Diffusion Language Models: Latent-Space Caching for Fast Generation
Show answer and example
Answer: Reusing expensive latent outputs saves anchor calls, but changing states make anchors stale. A cheap conditioned fusion predicts and gates corrections for a smaller denoiser. It cannot guarantee full recovery, and local tests omit full generation.
Illustrative example: Adjust an early rough sentence representation once later words become known.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
How does gated stale-cache correction differ from reuse unchanged?
Time-Anchored Diffusion Language Models: Latent-Space Caching for Fast Generation
Show answer and example
Answer: It combines c_t and the old anchor to adjust stale information. Refresh may still be needed; fusion is not an exact substitute for every anchor call.
Illustrative example: A changed denoising state can revise old features instead of treating them as fresh.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
What boundary checks validate TADM's g and Δ fusion?
Time-Anchored Diffusion Language Models: Latent-Space Caching for Fast Generation
Show answer and example
Answer: Use h_t′+g⊙Δ. Zero g or zero Δ recovers the anchor, not zero. Local cache correction does not load DiffusionGemma-26B.
Illustrative example: Turning the gate off keeps the original anchor.
Sections in the Chinese source: 核心公式, 复现边界. The example illustrates the mechanism.
Basics
Why window only the draft?
Windowed-MTP: Removing the Full-Context Draft-KV Tax at Million-Token Context
Show answer and example
Answer: Draft KV reads can bottleneck long contexts. Retain sinks and recent draft history while the full target verifies, changing proposal cost and acceptance.
Illustrative example: Draft might read recent 64 and a few sinks while target reads all 16K.
Sections in the Chinese source: 背景与主要改动. The example illustrates the mechanism.
Core mechanism
Why can shrinking only Windowed-MTP's draft context save reads without changing the target distribution?
Windowed-MTP: Removing the Full-Context Draft-KV Tax at Million-Token Context
Show answer and example
Answer: Draft reads sinks and recent tokens; the full target still sees all history and corrects proposals. The window affects draft cost and acceptance, not the exact target definition. Cutting target context changes the guarantee; local latency does not transfer to production.
Illustrative example: Target can reject a wrong draft caused by missing old evidence, provided its own history stays complete.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
Why doesn't a large reduction in draft keys yield the same TPOT reduction?
Windowed-MTP: Removing the Full-Context Draft-KV Tax at Million-Token Context
Show answer and example
Answer: Draft forwards and target checks still cost time, and time per output depends on accepted length AL. Lower acceptance can offset savings.
Illustrative example: Cheaper verification that accepts fewer tokens may yield little throughput gain.
Sections in the Chinese source: 核心公式, 本地复现. The example illustrates the mechanism.
Implementation details
Should overlaps between sinks and recent tokens duplicate KV?
Windowed-MTP: Removing the Full-Context Draft-KV Tax at Million-Token Context
Show answer and example
Answer: No. Use the union without duplicates. Only draft indices change; target retains the full cache.
Illustrative example: A position in both sets appears once in draft attention.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Topic guide and reading order
Start with this question¶
How can we reduce serial token generation while being explicit about whether the output distribution changes?
Takeaway¶
Speculative decoding proposes several tokens with a cheap draft and verifies them in one target pass. Correct acceptance and residual correction preserve the target distribution. Diffusion language models instead denoise multiple positions; dQwen3.5 changes a causal backbone with recurrent layers to bidirectional propagation. Few-step generation and shared iterative networks change refinement schedules. These are changes at different levels.
Example¶
A draft may propose four tokens and the target accept only the first two. The rest require correction and continued generation. Speed depends on acceptance and verification cost. A diffusion pass may update several positions, but multiple denoising passes can be needed. Compare full-output quality and end-to-end time, not tokens written in one pass.
Common pitfall¶
Exact speculative sampling depends on the full acceptance and correction algorithm; it does not apply to arbitrary multihead drafts. Diffusion or distilled models are not automatically equivalent to the autoregressive target. Markov drafts, reference kernels, or call counts are not actual large-model speedups.
Reading order¶
Start with the proposal, verification, and corrected sampling in Fast Inference. Then read dQwen3.5 on its bidirectional hybrid and parallel decoding. Distinguish rescheduling from training a new generative mechanism before exploring self-drafts, caching, and few-step refinements.
- Fast Inference from Transformers via Speculative Decoding · Original paper
- dQwen3.5: Adapting Hybrid Language Models into Bidirectional Diffusion Language Models · Original paper
Check your understanding¶
Can “speculative decoding preserves the distribution” also describe changing an autoregressive backbone to diffusion?
Show suggested answer
No. Speculation reschedules computation under a fixed target distribution with acceptance and correction. Diffusion changes propagation, training, and decoding. It requires independent quality evaluation and cannot inherit that guarantee.