STEPQuant: Spatiotemporal quantization of recurrent linear-attention state

简体中文

L1 lifetime-aware allocation and dual-axis scale fitting, not SGLang kernels or whole-model throughput.

Paper information

Field	Details
Paper	STEPQuant
Institution	Zhejiang University (first author).
First public release	2026-09-29 (arXiv v1)
Author code	Dreamer-Toby/STEPQuant.
Local adapter or method key	stepquant
Local reproduction code	src/auto_research/foundation_latest_20260930.py

Paper method and evidence

Each concurrent request retains a fixed-size delta-rule state, and quantization errors compound. Estimate remaining state lifetime and key-row impact, allocate more precision to important blocks under a fixed average budget, and fit row/column scales.

flowchart LR
  S[Recurrent state] --> L[Remaining lifetime and key-row impact]
  L --> B[Allocate bits within budget]
  B --> Q[Quantize with row and column scales]
  Q --> R[Next recurrent update]

Original paper figure

Figure 1 motivates quantization by linking state memory, lifetime and error. From the original paper; copyright remains with the authors.

Core equation

Use normalized lifetime \(w_t=(T-t)/\sum_j(T-j)\) and solve \(\min\sum_i w_i e_{i,b_i}\) exactly over candidate widths subject to average bits. Quantize with the geometric combination of key-row and value-column scales, not uniform INT8.

Reported results

For Qwen3.8-27B and Kimi-Linear-48B-A3B-Instruct, the paper reports nominal 6-bit state near FP32, over 5× state compression and up to 68.7% lower serving memory. Not local results.

Local reproduction

Three-seed diagnostic verifies bit budgets, finite scales, state MSE and CUDA forward/backward. Sanitized A100 receipt. No reproduction of throughput or model accuracy.