TransMem: Transforming Hidden States into Memory for Large Language Models¶
Fixed-context NumPy diagnostic, not paper-scale training.
Paper information¶
| Field | Details |
|---|---|
| Paper | Paper or original source |
| Institution | No affiliation disclosed; first author Haodong Lei. |
| First public release | 2026-07-31 (arXiv v1) |
| Author code | No public author code found as of 2026-08-24. |
| Local adapter or method key | transmem |
| Local reproduction code | src/auto_research/reproductions/transmem/ |
Paper method and evidence¶
Convert sparse historical hidden states from a frozen backbone to reusable parametric memory. Learn a gate by evidence-conditioned self-distillation.
flowchart LR
A[Frozen hidden states] --> B[Create reusable memory]
B[Create reusable memory] --> C[Evidence-conditioned gate]Original paper figure¶
Figure shows hidden-state memory transformation. From the original paper; copyright remains with the authors.
Core equation¶
\[
h'_t=h_t+g_t\odot T(H_{sparse})
\]
Reported results¶
Paper: LoCoMo +11.58–29.25 F1, HotpotQA +10.20–13.03 F1, MemoryAgentBench 29.54%→40.00%.
Local reproduction¶
Same 64 examples and seed, recent-window attention 0.0781, TransMem 0.2031, +12.50 percentage points. Metrics, batch index. No original checkpoints, custom CUDA kernel or full benchmark.