HAWK: Rethinking Multimodal Drafting for Speculative Decoding

简体中文

L1 diagnostic of layer mixing and shifted target isolation. No specialized decoding kernel.

Paper information

Field	Details
Paper	arXiv v1
Institution	The paper's first page does not list an affiliation for the first author.
First public release	2026-09-30 (arXiv v1)
Author code	No public author implementation found as of 2026-10-03.
Local adapter or method key	hawk
Local reproduction code	src/auto_research/foundation_latest_20261003.py

Paper method and evidence

Learn a mixture of hidden states from multiple target layers to supply compressed visual features to a shallow drafter. Train on the shifted trajectory after the drafter's own proposals.

flowchart LR
  H[Target hidden states across layers] --> M[Learned visual feature mixture]
  M --> D[Shallow drafter]
  D --> S[Train against shifted target trajectory]

Original paper figure

Figure 1 shows the training process. From the original paper; copyright remains with the authors.

Core equation

\(h_d=\sum_l softmax(a)_l h_l\); supervision comes from the target distribution on the shifted trajectory.

Reported results

The authors report acceptance rates and lossless speedups for speculative decoding with LVLMs.

Local reproduction

Compare with the mechanism disabled/default state. The three-seed diagnostic is diagnostic_only=true; no formal relative gain or capability ranking.