Foundation model papers and materials¶
简体中文 · English
This index is built from docs/research-manifest.json, generated from the shared paper specifications.
Maintain metadata in packaged paper.yaml files and per-domain catalog specifications, not generated pages.
Open a paper's write-up for background, architecture, equations, reported results, and local evidence.
Implemented papers and research materials¶
| Topic | Method | Institution and first publication | Author code | Local entry point |
|---|---|---|---|---|
| Inference and systems efficiency | XLA from the Trenches: Correctness on TPU Workloads at Scale | Google, 2026-10-31 | Public HLO isolation and reference evaluator are upstream components; OpenXLA uses Apache-2.0. The internal agent harness, TPU tools, production graphs, and micro-fusion debugging pipeline are not public; the complete implementation is not released | Planned key xla-trenches — Not reproduced due to cost (review complete); research material only, no executable entry. |
| Multimodal foundation models | GAVAGAI: Referential Alignment for Infant-Scale Vision-Language Models via Balanced Optimal Transport | St Thomas’ College of Engineering and Technology, 2026-10-30 | No verifiable method implementation found in fixed v1, the abstract page, or method/author searches; not found/not yet released | gavagai — Low-cost component only; full experiments not reproduced. |
| Pretraining and data | ProLand: A New Infeasible Manifold Optimization Method for LoRA Fine-Tuning | Shanghai Jiao Tong University, 2026-10-30 | No verifiable method implementation found in fixed v1, the abstract page, or method/author searches; not found/not yet released. Muon or other LoRA repositories are not this method's code | proland — Low-cost component only; full experiments not reproduced. |
| Inference and systems efficiency | Folding Attention Biases into Keys Makes Compacted KV Caches Run on Standard Kernels | Independent Researcher, 2026-10-27 | No verifiable method repository or code revision found in the reviewed v1 text or abstract page. | fold-bias — Low-cost component only; full experiments not reproduced. |
| JEPA and world models | GUI-JEPA: Candidate-Conditioned JEPA Latents as a Fast Action Policy for Mobile GUI Agents | Shanghai Jiao Tong University / Nanyang Technological University, 2026-10-27 | bcwang-sjtu/GUI-JEPA, reviewed at 07e82c786d7ca3ace67385fdf59b9a52f8982fd6, contained only a placeholder README. No method code or LICENSE was found. No code or weights were copied. The paper uses CC BY 4.0. |
gui-jepa — Low-cost component only; full experiments not reproduced. |
| Attention and long context | Multi-State Linear Attention with Generalized Second-Order Routing | Princeton University, 2026-10-27 | No verifiable author implementation was found in the v1 paper or abstract page; no source revision can be pinned. | merlin — Low-cost component only; full experiments not reproduced. |
| Attention and long context | Select-TTT: Query-Conditioned Selective Fast Weight Updates for In-Place Test-Time Training | University of Science and Technology of China, 2026-10-27 | Appendix D promises a release on publication; the reviewed v1 has no verifiable project link or method implementation. | select-ttt — Low-cost component only; full experiments not reproduced. |
| JEPA and world models | SWIM: Learning What Matters from Behaviors for JEPA World Models | Boston University / Unity Technologies, 2026-10-27 | No verifiable SWIM repository or code revision found in the reviewed v1 text or abstract page. LeWM and other baseline repositories are not SWIM source code. | swim — Low-cost component only; full experiments not reproduced. |
| Inference and systems efficiency | ResCom: Activation Residual Compensation for Large Language Model Quantization | Institute of Automation, Chinese Academy of Sciences, 2026-10-25 | No method repository was found in the reviewed v1 text, appendix or arXiv page. Baseline repositories such as SERQ are not ResCom source. | rescom — Low-cost component only; full experiments not reproduced. |
| JEPA and world models | DT-JEPA: Learning to Predict Future Representations for Decision-Making | Southeast University, 2026-10-23 | No author DT-JEPA source or method license was verified in v1 or its official links. TD-JEPA and similar names are not substitutes. The paper is CC BY 4.0. | dt-jepa — Low-cost component only; full experiments not reproduced. |
| System One / JEV | EditJev: System One Routing for Image Editing Models | Peking University, 2026-10-23 | No author EditJev method repository or source license verified. Candidate-editor repositories are not the router source; the paper has an arXiv non-exclusive license. | editjev — Low-cost component only; full experiments not reproduced. |
| System One / JEV | PathJev: Decision-Native Vision–Language Modelling for Histopathology | University of Naples Federico II, 2026-10-23 | CosmoIknosLab/PathJev at 1f5fa8d29220fbdbc31496484170e85fe83d4bea, Apache-2.0, includes implementation, inference, analysis, configs, tests and results. PathJev-P weights are available. Paper CC BY-NC-ND 4.0; MedGemma and data terms are separate. |
Planned key pathjev — Not reproduced due to cost (review complete); research material only, no executable entry. |
| World models | Inside-JEPA: From Prediction to Relational Grounding in Latent World Models | Australian Institute for Machine Learning, Adelaide University, 2026-10-22 | No author method repository, SHA or code license verified. The paper is CC BY 4.0; “code is distributed” describes the representation, not an open-source release. | inside-jepa — Low-cost component only; full experiments not reproduced. |
| System One / JEV | Jev-DLC: Decision-Level Continual Learning from Typed Feedback | The Hong Kong University of Science and Technology (Guangzhou), 2026-10-22 | No Jev-DLC method repository was found in the paper or its official links. Describing an artifact chain does not establish a public release. OpenJev is a referenced base system, not the method's source. No method revision or license was verified. | jev-dlc — Low-cost component only; full experiments not reproduced. |
| Inference and systems efficiency | LazyFlash: Forward-Dense, Backward-Sparse Attention for Accelerated Long-Context Fine-Tuning | Microsoft, 2026-10-22 | No verifiable LazyFlash method repository or code license in reviewed v1. FlashAttention is an upstream library, not the author's LazyFlash source. | Planned key lazyflash — Not reproduced due to cost (review complete); research material only, no executable entry. |
| Efficient attention | OpenAttention: One Percent Is Enough with Tail Proxies | Sogang University, 2026-10-22 | hyunwoo137/OpenAttention, Apache-2.0 at 6e43d4fc8c0e95b26d17d198a4545c627b2d6931. |
openattention — Low-cost component only; full experiments not reproduced. |
| Sequence compression | Rethinking Sequence Compression with Test-time Regression | The Chinese University of Hong Kong, 2026-10-22 | Sphere-AI-Lab/ttr-compressor at 29f643f1348a4b3893f4740433b1fc8b8c39b2e7; project page. No clear core license was verified, and third-party licenses do not cover the original core. Independent local implementation. The repository reports October 20 without an exact verified paper timestamp; this page uses arXiv v1. |
ttr-compressor — Low-cost component only; full experiments not reproduced. |
| Policy distillation theory | Reverse-KL Policy Distillation: Statistical Guarantees and Limits | University of Virginia, 2026-10-22 | No public author repository in the paper or arXiv entry; no source license applicable. | rkl-policy-distillation — Low-cost component only; full experiments not reproduced. |
| Visual representations and world models | ALPS: Visual State Encoders for World Models | University of California, Santa Cruz, 2026-10-21 | UCSC-VLAA/ALPS, Apache-2.0, 30ab44b9fcdf3253fc67f5a53e31ea5141eb45c1; training/inference/evaluation source released 2026-10-22 01:45:01 UTC. Public weights were not downloaded. |
alps — Low-cost component only; full experiments not reproduced. |
| Interpretability | Building Auditable AI Text Detectors with Sparse Autoencoders | University of Pennsylvania, 2026-10-21 | Official DeepMind repository, Apache-2.0 at 52ecc87775154b35d4550911de22faad4fc68f40; model/data terms separate. Created 2026-10-24 13:32:05 UTC. An Oct 22 commit timestamp does not establish public release that day. |
detector-scope — Low-cost component only; full experiments not reproduced. |
| MoE routing | MoEZip: Routing-Aware KV Cache Compression for Sparse Mixture-of-Experts LLMs | Korea University, 2026-10-21 | kmswin1/MoEZip at 197959dbae2adfab7ec233f584ce951b827e04cc contains only README and Apache-2.0 LICENSE, no implementation. README examples are not executable release evidence. |
moezip — Low-cost component only; full experiments not reproduced. |
| Model compression and quantization | Right On Q: Recovering Reasoning under Quantization via Self-Generated Trajectories | University of Michigan, Ann Arbor, 2026-10-21 | No verifiable author repository in v1 or arXiv. EvalScope is not this implementation. No software license verified; arXiv distribution terms are separate. | Planned key right-on-q — Not reproduced due to cost (review complete); research material only, no executable entry. |
| System One / JEV | When Multiple Choices Are Correct: Asymmetric Probability Allocation in Jev and Other Decision Models | Independent Researcher, 2026-10-21 | Rushab1/gr-jev, MIT, tag arxiv-v1 at 7b161f96cd0ff1842845fc66f981d1471cc18151. Saved model responses were not released. |
jev-multicorrect — Low-cost component only; full experiments not reproduced. |
| JEPA and world models | LocaLeJEPA: Maximum Entropy Representations under Geometric Priors | University of Pennsylvania, 2026-10-20 | alessioborgi/LocaLeJEPA, MIT, formula review at 771361c18d37e65df3dc188761ffb26751ce4e37. |
locale-jepa — Low-cost component only; full experiments not reproduced. |
| Attention and long context | RACE: RoPE-Aligned Attention Compression Based on Query-Key Activation Energies | MIT CSAIL, 2026-10-20 | No verifiable author repository or revision in the paper or abstract. Related-work repositories are not method source. | race-rope-compression — Low-cost component only; full experiments not reproduced. |
| Attention and long context | Stratified Sparse Attention: Reducing the Sparse-to-Dense Gap at Extreme Sparsity | Rice University, 2026-10-20 | The claimed SSA-Paper-Code returned 404 on 2026-10-21. No verified code or revision. | stratified-sparse-attention — Low-cost component only; full experiments not reproduced. |
| Multilingual modeling and tokenization | DeepSEA: A Multilingual Evaluation Benchmark for Southeast Asia | Google Research, 2026-10-19 | google-research/deep-sea at 7c3abc06824c8cb634930ab6049176da30cfa0e7; scoring code Apache-2.0, separate data terms including some non-commercial/share-alike restrictions. |
deepsea — Low-cost component only; full experiments not reproduced. |
| Inference and systems efficiency | FLINT: Fixed-budget Late Interaction | University of Illinois Urbana-Champaign, 2026-10-19 | hmk55555/FLINT, MIT, 3b3f3e37d34e32ca5c92cc5b66e3f778ae4210ec; checkpoint collection, not downloaded. |
flint — Low-cost component only; full experiments not reproduced. |
| Attention and long context | Kronecker Delta Rule for Long Context Linear Attention | Johannes Kepler University Linz, 2026-10-19 | The abstract announces code/models, but no verifiable repository, release location or license was found in the reviewed abstract/HTML/PDF. lm-engine/FLA are not the method source. | kronlstm — Low-cost component only; full experiments not reproduced. |
| Pretraining and data | T-MUON: Tensorizing Muon via t-SVD for Large Language Model Pre-training | University of California, Santa Barbara, 2026-10-18 | No verifiable source repository found in the paper or author sources during this review. Code license unverified. | t-muon — Low-cost component only; full experiments not reproduced. |
| Model architecture | Muse Glimmer 30B: An On-Device Agentic Model | Meta Superintelligence Labs, 2026-10-17 | The official model repository is Apache-2.0. Weights, configuration, tokenizer, and chat template were verified. Complete pretraining and OPD training source and data were not verified. Weight availability and training reproducibility are tracked separately. | Planned key muse-glimmer-30b — Not reproduced due to cost (review complete); research material only, no executable entry. |
| Pretraining and data | BMuon: Budgeting Muon's Spectral Geometry for Structured Compressibility | Inria, 2026-10-16 | The official repository had only a README at review time. It says the implementation and reproduction scripts will be released later. No implementation or license was verified. | bmuon — Low-cost component only; full experiments not reproduced. |
| System One / JEV | Jev-as-a-Judge for Brazilian Portuguese Information Retrieval: Relevance Assessment and Reranking | Câmara dos Deputados, 2026-10-16 | No author implementation repository was found as of 2026-10-19. The TypeSafe API or SDK does not make the paper code open source. | jev-portuguese-ir — CPU evaluation interface; does not run a model. |
| Multimodal foundation models | Persistent Planar Memory for Video World Models | Tsinghua University, 2026-10-16 | arXiv lists the project page above. The page could not be retrieved in this review, so runnable author code and license could not be verified. A project page alone does not establish release or non-release. | Planned key ppmem — Not reproduced due to cost (review complete); research material only, no executable entry. |
| Inference and systems efficiency | Sink-Sparse Hybrid Attention for Efficient Inference in T2I/T2V Diffusion Transformers | Xi'an Jiaotong University, 2026-10-16 | No repository is listed in the abstract or v1 text. No verifiable author implementation or code license was found. | sink-sparse — Low-cost component only; full experiments not reproduced. |
| Multimodal foundation models | AdaLoop: Thinking with the Vision Encoder | Harvard University, 2026-10-15 | No verified author implementation or license was found. Code for other cited models is not AdaLoop source. | adaloop-vision — Low-cost component only; full experiments not reproduced. |
| Pretraining and data | Architecting for Hardware Diversity: Meta's End-to-End Co-Design for High-Performance AI Training on AMD GPUs | Meta, 2026-10-15 | The paper says changes were upstreamed to PyTorch, Triton, and ROCm. No separate complete reproduction repository found. Those projects are not in their entirety this paper's source. | Planned key hardware-diversity — Not reproduced due to cost (review complete); research material only, no executable entry. |
| Pretraining and data | BitSparse: Compact Activation Storage for ReLU Feed-Forward Networks | University of Cambridge, 2026-10-15 | Maxzhu123/Bitmask_Sparsity is public to read. The license is unverified; copying is not assumed permissible. No upstream code is vendored. | bit-sparse — Low-cost component only; full experiments not reproduced. |
| KV caching | Compression Does Not End at Selection: Post-Selection Positional Geometry for Sparse KV Caches | National University of Defense Technology, 2026-10-15 | No released method repository was found. Appendix A.1 says implementations and configurations will be released in the future. Equations were implemented independently, with no vendored code. | rope-relay — Low-cost component only; full experiments not reproduced. |
| Model architecture | Designing Transformers for Continuous Diffusion Language Models | Dots Studio, Xiaohongshu, 2026-10-15 | No verified author repository or license found. The local implementation follows the paper independently. | langdit — Low-cost component only; full experiments not reproduced. |
| Inference and systems efficiency | DropKV: Decoupling Residual-Output Perturbation for Near-Optimal KV-Cache Eviction | University at Albany, SUNY, 2026-10-15 | AozhongZhang/DropKV, Apache-2.0, also integrated in NVIDIA kvpress. Local implementation independently follows the formulas. | drop-kv — Low-cost component only; full experiments not reproduced. |
| Multimodal foundation models | EgoP²C: Internalizing Plan-Perceive-Compose for Spatial Reasoning in Egocentric Video | Arizona State University, 2026-10-15 | The arXiv abstract and v1 text list no repository. No verifiable author implementation or code license was found. | ego-p2c — Low-cost component only; full experiments not reproduced. |
| Inference and systems efficiency | Fractional Power-of-Two Quantization for Efficient and Effective Multiplier-Free LLM Inference | Seoul National University, 2026-10-15 | No verified author repository or license was found; the abstract and full text did not list one. | fpot — Low-cost component only; full experiments not reproduced. |
| Multimodal foundation models | Gen4P: Video Generators are 4D Perceivers | Nanyang Technological University, 2026-10-15 | The official repository has a README and assets only at review time, with Installation/Usage marked WIP. Runnable training/inference source and its license were not verified. | Planned key gen4p — Not reproduced due to cost (review complete); research material only, no executable entry. |
| Multimodal foundation models | LAMP: Layerwise Adaptive Mixing of Predictors for Training-Free Few-Shot Vision-Language Adaptation | Taizhou Institute of Science and Technology, Nanjing University of Science and Technology, 2026-10-15 | Abstract and full text list no code link. No verified repository or license found. | lamp — Low-cost component only; full experiments not reproduced. |
| Pretraining and data | LCT: Losslessly Compressed Tensors with GPU friendly execution for Training | University of Cambridge, 2026-10-15 | Maxzhu123/Losslessly_Compressed_Tensors, MIT. Independent local implementation; the upstream Python 3.14 environment was not installed. | lct — Low-cost component only; full experiments not reproduced. |
| JEV and single-step decisions | MJev-Spatial: System One Spatial Intelligence via Native Attention Matching | Unspecified (no first-author affiliation; released publicly by MindLab), 2026-10-15 | Author repository, Apache-2.0; the 4B checkpoint was not downloaded. | mjev-spatial — Low-cost component only; full experiments not reproduced. |
| Byte-level models | Patch where the text is new: a compression view of where byte-level models should spend compute | Independent Researcher, Chile, 2026-10-15 | nicoveraz/segresearch, MIT. The paper points to paper2/ and scale/. |
novelty-patching — Low-cost component only; full experiments not reproduced. |
| Inference and systems efficiency | Prism-1: Position- and Rival-Independent Scoring for System One Models | Nanyang Technological University, 2026-10-15 | No verifiable author repository or license found. Repositories of other cited models are not Prism-1 code. | prism-1 — Low-cost component only; full experiments not reproduced. |
| Model architecture | Rethinking Routing Exploration in Modern Mixture-of-Experts | ETH Zurich, 2026-10-15 | BingcongLi/MirrorMoE, MIT. The local implementation follows the equations independently and does not import Megatron. | mirror-moe — Low-cost component only; full experiments not reproduced. |
| Multimodal foundation models | Retrofitting Recurrent Vision-Language Models from Pretrained Backbones | University of Tübingen, 2026-10-15 | Official project page says Code (coming soon). No verifiable implementation or license found at review. | rr-vlm — Low-cost component only; full experiments not reproduced. |
| Inference and systems efficiency | RoleQ: Role-Aligned Scales for Converting Language Models to Ternary | Harvard University, 2026-10-15 | No author-released method implementation or verifiable license was found. The local component independently implements the paper's equations. | roleq — Low-cost component only; full experiments not reproduced. |
| Inference and systems efficiency | SoftCache: Adaptive Step Size Caching for Diffusion Transformers | Tel Aviv University, 2026-10-15 | The official project page was inaccessible during review. No verified author repository or license was found. | softcache — Low-cost component only; full experiments not reproduced. |
| Pretraining and data | When Does Muon Actually Win? A Paired Data-, Time-, and Tuning-Budget Benchmark | First author's institution not stated in the paper, 2026-10-15 | No verifiable repository or license for the authors' experiments was found during this review. Citing upstream Muon is not a release of this experiment's code. | muon-paired-budget — CPU evaluation interface; does not run a model. |
| Inference and systems efficiency | When the KV Cache Is Slower Than the GPU: Scaling Remote KV Tiers for Context-Parallel Prefill | Multiverse Computing, 2026-10-15 | Public author/upstream code. Relevant implementations are public as the authors' upstream LMCache PRs. Public PRs do not establish that all changes are merged or that the entire cluster experiment is available to reproduce. | Planned key remote-kv-tiers — Not reproduced due to cost (review complete); research material only, no executable entry. |
| Pretraining and data | Dataset-Level Regularization for Joint-Embedding Self-Supervised Learning | European Molecular Biology Laboratory (EMBL), 2026-10-14 | No release found. The reproducibility statement only says code is planned; referenced SimCLR and SIGReg repositories are not the authors' implementation for this paper. | oob-regularization — Low-cost component only; full experiments not reproduced. |
| JEPA and world models | EgoJEPA: Beyond Retargeting for Dexterous Manipulation from Egocentric Videos | Mohamed bin Zayed University of Artificial Intelligence, 2026-10-14 | No verifiable author source repository was found. The GitHub button on the project page points to the paper, not a code release. | Planned key egojepa — Not reproduced due to cost (review complete); research material only, no executable entry. |
| Multimodal foundation models | ESC: Escaping 2D Diffusion Ambiguity with Geometric Constraints for Consistent Novel View Synthesis | FAIR at Meta / Mila - Québec AI Institute / Université de Montréal, 2026-10-14 | The paper provides no verified author repository URL, and none was found or confirmed in this review. Qwen or DA3 base-model repositories are not ESC implementations. | Planned key esc — Not reproduced due to cost (review complete); research material only, no executable entry. |
| Pretraining and data | Memory-Efficient Subspace Optimization with Residual Correction: From Momentum SGD to Muon | Nanjing University / Alibaba Token Hub, 2026-10-14 | No dedicated author repository for SMRC/PMRC was found. Dependencies such as Minimalist_LLM_Pretraining do not establish a code release for the new method (v1 checked 2026-10-15). | smrc-pmrc — Low-cost component only; full experiments not reproduced. |
| Multimodal foundation models | One-Step Prediction Fidelity Is a Poor Proxy for Planning Performance in JEPA World Models | Institution not stated in the paper, 2026-10-14 | No separate author implementation for this evaluation was found when v1 was checked on 2026-10-15. LeWorldModel is used in the paper but is not a separate code release for this study. | jepa-prediction-planning — CPU evaluation interface; does not run a model. |
| Model architecture | Open-world AI: a Tale of Three Requirements | FAIR, Meta, 2026-10-14 | MemoryMosaics repository releases Mosaics+ and two-stage configurations under the Apache-2.0 license. Local code is independent and does not copy upstream source. | memory-mosaics-plus — Low-cost component only; full experiments not reproduced. |
| Inference and systems efficiency | PN-Mix: Pair-Norm Guided Mixed-Precision Key Cache Quantization for Efficient Long-Output LLM Inference | Seoul National University, 2026-10-14 | No author PN-Mix code found as of 2026-10-15. Kitty, linked in the paper, is a comparison baseline, not PN-Mix code. | pn-mix — Low-cost component only; full experiments not reproduced. |
| System One / JEV | Selective Reasoning for Typed Decision Models: When to Start, Read, and Stop | University of Electronic Science and Technology of China, 2026-10-14 | Yes: author repository, pinned v1.0 release, commit 26795854fe148e7a4caa37ef6cc051431b8b8790 |
selective-reasoning — CPU evaluation interface; does not run a model. |
| JEPA and world models | Time-JEPA: Grounding Video World Models with Physical Time Series | Harbin Institute of Technology, 2026-10-14 | Official repository includes src/app/configs/tests and evaluation scripts. MIT and Apache-2.0 files apply within their respective scopes. The full upstream training was not run. | time-jepa — Low-cost component only; full experiments not reproduced. |
| System One / JEV | The Company a Question Keeps: Auditing Open Jev-Style Models for Retrieval | Tsinghua University, 2026-10-10 | No public download for the paper artifact was found as of 2026-10-14. The paper describes saved fixtures, predictions, and analysis code; the evaluated Laya/Kev/Nimble/Von repositories are not this artifact. | jev-request-audit |
| System One / JEV | Does System One Transfer? Calibrated Typed-Decision Models, Label Budgets, and Translation for Indonesian Regional Languages | Universitas Budi Luhur, 2026-10-09 | Code, preregistration, and all predictions, pinned to a8499c2b71815ca512563dbb7fdc5c51dc783f6a |
jev-regional-transfer |
| Inference and systems efficiency | DualQuant: Dual-scale Calibration-free Weight-and-Activation Quantization for LLMs | d-Matrix, 2026-10-09 | Yes: authors' code. | dualquant |
| Multimodal foundation models | EV-JEPA: Elastic V-JEPA for Any Device and Cross-Device Inference | Meta Reality Labs Research, 2026-10-09 | Not found: v1 did not link official implementation for this method as checked on 2026-10-14. This is not a claim that no code exists anywhere. | Planned key ev-jepa — Not reproduced for now due to cost; validation pending, no executable entry. |
| Model architecture | Multi-Stream Attention Residuals | Shanghai Jiao Tong University, 2026-10-09 | The author-provided repository returned 404 on 2026-10-12; no usable author source was found. | msar |
| Attention and long context | Not All Dimensions Are Equal: Outlier-Aware Attention Linearization with Grouped Quadratic Kernels | The Chinese University of Hong Kong, Shenzhen, 2026-10-09 | Author repository; no license was found in the reviewed snapshot. | oal |
| Model architecture | On the Design Space of Continuous Diffusion Language Models | University of Oxford / Apple (work done at Apple), 2026-10-09 | No public author implementation found as of 2026-10-13. The paper links only generic tiktoken code. | doublecat-semicat |
| Attention and long context | Orthros: Phase-Aware Heterogeneous Attention for Efficient Transformers | The Hong Kong Polytechnic University, 2026-10-09 | No public author implementation found as of 2026-10-12 | orthros |
| Inference and systems efficiency | ReplaySSM: Cache SSM Inputs, Not State | Princeton University, 2026-10-09 | ReplaySSM, Apache-2.0 | replayssm |
| Multimodal foundation models | SmolWAM: A Latent World-Action Model for Robotic Manipulation on Edge Devices | ML Collective, 2026-10-09 | Yes: authors' code. | smolwam |
| Continuous reasoning | CoTS: Competitive Token Selection for Latent Reasoning in Large Language Models | Xiaomi AI Lab, 2026-10-08 | OkinoYu/CoTSReasoning; no clear license identified, so implemented from equations without vendoring. | cots — Low-cost component only; full experiments not reproduced. |
| Generative models | Few-Step Generation via Data-Space Iteration | ByteDance Seed, 2026-10-08 | Appendix E, Algorithms 1–2 publishes core PyTorch code, not a complete training repository; no complete external author repository found as of 2026-10-11. | data-space-iteration |
| Multimodal foundation models | LeWAM: End-to-End World and Action Modeling with JEPAs | UC San Diego, 2026-10-08 | Yes: authors' repository | lewam |
| Multimodal foundation models | MetaEncoder: Exploring the Limit of Bi-Encoders for Multimodal System One Decision Making with Natural Language Interface | Meta AI, 2026-10-08 | Official evaluation code, Apache-2.0; released weights. Full training source/data are not implied. | Planned key metaencoder — Not reproduced due to cost (review complete); research material only, no executable entry. |
| Training optimization and stability | Rounding in Preconditioner Space: Redesigning 4-bit AdamW Optimizer-State Quantization | University of California, Berkeley / Nubank, 2026-10-08 | nubank/adamw4bit, Apache-2.0. | adamw4bit |
| Sparse experts and routing | Smoothing the Top-k Exposure Boundary for Sparse Mixture-of-Experts | Shanghai AI Laboratory / Shanghai Jiao Tong University, 2026-10-08 | No public author implementation found; third-party code is not official. | elastic-expert-routing |
| Inference and systems efficiency | SparseDecoding: Decoding-Aware Pruning for Accurate and Efficient LLM Inference | Westlake University, 2026-10-08 | Not yet released; the project page says “Code (soon).” This repository implements it independently. | sparse-decoding |
| System One / JEV | Specialized Decision Models vs. General-Purpose LLMs: Benchmarking Jev Across Knowledge, Reasoning, and Multilingual Tasks | Dalian Maritime University, 2026-10-08 | No author code found as of 2026-10-10; TypeSafe Jev weights are not public. | jev-capability |
| Inference and systems efficiency | VFold: Symmetry-Aware Cross-Layer Value Cache Compression | Johns Hopkins University, 2026-10-08 | Released; local code is independent. | vfold |
| Video understanding | Video Encoders Built on Image Representations | Stanford University, 2026-10-05 | No implementation repository linked in the official arXiv v1 HTML or abstract. This does not establish worldwide absence. | video-encoder-selection — Low-cost component only; full experiments not reproduced. |
| Optimizers | AF-Muon: An AdamW-Free Muon Optimizer for Tied-Embedding Models | First author's affiliation not listed on the first page, 2026-10-01 | No public author implementation found as of 2026-10-03. | af-muon |
| Structural pruning | IrekoGPT: Turning Structured Pruning into Post-Hoc Slimmable LLMs | University of Modena and Reggio Emilia, 2026-10-01 | aimagelab/IrekoGPT. | irekogpt |
| Jev | LLM2Jev: LLMs Are Already Jev-Style Decision Models -- When and How to Fine-Tune Them | Microsoft (based on author email), 2026-10-01 | No public author implementation found as of 2026-10-03. | llm2jev |
| Multimodal language models | MWOP: Modality-aware Width-wise Operation Pruning for Efficient MLLMs | First author's affiliation not listed on the first page, 2026-10-01 | No public author implementation was found as of 2026-10-03. | mwop |
| Multimodal embeddings | Omni-Embed-Mini: Binding Modalities Without Forgetting via Dense Distillation | First author's affiliation not listed on the first page, 2026-10-01 | No public author implementation found as of 2026-10-03. | omni-embed-mini |
| Long context | Role-aware Heuristic Episodic Attention for Conversational LLMs | First author's affiliation not listed on the first page, 2026-10-01 | No public author implementation found as of 2026-10-03. | rea |
| Optimizers | TACO: Ternary Absolute-max Column-wise One-sparse Optimizer for LLM Fine-Tuning | University of Central Florida, 2026-10-01 | TACO_optimizer. | taco-optimizer |
| Pretraining and data | TESS: Scalable, Transferable Meta-network for Data Selection Requires a Different Loss (and Why the Obvious Choice is Problematic) | Nanyang Technological University, 2026-10-01 | No author code found in full text on 2026-10-04; Alpaca data is not method source. | tess |
| Video VLMs | VETO: Video Efficient Token Optimization for Vision Language Models | National Taiwan University, 2026-10-01 | No public author implementation was found as of 2026-10-02. | veto |
| Multimodal reasoning | HAWK: Rethinking Multimodal Drafting for Speculative Decoding | First author's affiliation not listed on the first page, 2026-09-30 | No public author implementation found as of 2026-10-03. | hawk |
| System One / JEV | Chinese-Jev: Bringing System One Model to Chinese-Language Tasks | Fudan University, 2026-09-29 | No standalone repository released; the project page says models, data and code are planned. | chinese-jev |
| MoE | Cross-Entropy Guided Routing in Mixture-of-Experts Large Language Models | Bar-Ilan University, 2026-09-29 | No public author repository found as of 2026-10-01. | ce-guided-moe |
| Inference and systems efficiency | LeapQuant: Efficient Linear Attention with Accurate Recurrent State Quantization | UC Berkeley, 2026-09-29 | No author repository was provided as of 2026-09-30. | leapquant |
| Diffusion language models | On Trajectory-Aware Training for Masked Diffusion Language Models | Apple, 2026-09-29 | No public author repository found as of 2026-10-01. | pumba |
| Model architecture | Pretraining Latent Information Feedback Transformers with Teacher Supervision | Tel Aviv University, 2026-09-29 | Yes: dortirosh1/LIFT. | lift-feedback |
| Inference systems | SPLASH: Switching Parallel Layouts of Attention with Seamless Handoff for LLM Serving | Institute of Computing Technology, Chinese Academy of Sciences, 2026-09-29 | SPLASH-sglang. | splash |
| Inference and systems efficiency | STEPQuant: When and Where Errors Matter in Delta-Rule Recurrent State Quantization | Zhejiang University, 2026-09-29 | Dreamer-Toby/STEPQuant. | stepquant |
| Diffusion language models | Time-Anchored Diffusion Language Models: Latent-Space Caching for Fast Generation | The University of Texas at Austin, 2026-09-29 | No public author repository found as of 2026-10-01. | tadm |
| Model architecture | Triadic Linear Attention: Three-Dimensional Recurrent States for Long-Context Sequence Modeling | MIT / MIT-IBM Watson AI Lab, 2026-09-29 | Yes: OliverSieberling/TriadicLinearAttention | triadic-linear-attention |
| World action models | V-JEPA Policy: Building Effective World-Action Models on Predictive Visual Latents | Tsinghua University, 2026-09-29 | VJEPA-Policy | vjepa-policy |
| State-space models | Fractional State Space Transition for Long Sequence Modeling | Huawei Noah's Ark Lab, Montreal Research Center, 2026-09-28 | anasiri/frac-ssm. | frac-ssm |
| Model architecture | MS-GLA: Multi-Scale Gated Linear Attention for Addressing Representational Bottlenecks via Multi-Temporal Resolution | International Institute of Information Technology Hyderabad, 2026-09-28 | prasoondev/msgla | ms-gla |
| Model architecture | Telescopic Language Models | University of Cambridge, 2026-09-28 | No public author repository found as of 2026-09-30. | telescopic-lm |
| Public evaluation protocols | Words Speak Louder Than Order: A Behavioral Evaluation of Gemma 4 | Google, 2026-09-25 | Official author directory, pinned at fc468f2374540870042b20db1ec7e836ed205db7. The README declares Apache-2.0 and the directory LICENSE supplies its title and official link. |
gemma-priority-protocol — Low-cost component only; full experiments not reproduced. |
| Diffusion language models | dQwen3.5: Adapting Hybrid Language Models into Bidirectional Diffusion Language Models | University of Texas at Austin, 2026-09-17 | No public implementation repository was found as of 2026-09-19. | dqwen35 |
| Attention and long context | On-Demand Attention: Efficient Long-Context Decoding with Learned Recall | Southern University of Science and Technology, 2026-09-17 | No public author implementation found as of 2026-09-19. | oda |
| Speculative decoding | ASPIRE: Asynchronous Batch Self-Speculative Decoding | University of Southern California, 2026-09-16 | No public author repository found as of 2026-09-19. | aspire |
| Speculative decoding | ECHO: Early-layer Collaborative Hierarchical Orchestration with Bonus Logits in Speculative Decoding | Wuhan University, 2026-09-15 | ECHO | echo |
| Foundation model retrieval and ranking | How Calibration Content Shapes Attention-Based Reranking | UC San Diego, 2026-09-15 | No method repository found in v1 or exact-title search; code hash/license unverified. ICR/QRHeads baseline repositories do not release this method. Paper is CC BY 4.0. | interpolated-null-calibration — Low-cost component only; full experiments not reproduced. |
| Speculative decoding | LoopSpec: Pipelined Self-Speculative Decoding for Looped Transformers | Seoul National University / KAIST, 2026-09-15 | No public author repository found as of 2026-09-16. | loopspec |
| Reproducible training | OPEN-1B: A Fully Auditable Training Run | Gensyn, 2026-09-15 | Public training implementation; audit tool | open-1b-audit |
| Model architecture | Persistent Recurrent Memory Between Transformer Layers Improves Language Model Generalization | FITec Labs / Ericsson São Paulo, 2026-09-15 | No public author repository found as of 2026-09-16. | persistent-recurrent-memory |
| Agent reasoning | AgentKV: Phase-Aware KV Eviction for Agentic LLMs | University of Cambridge, 2026-09-14 | Yes: LiuTaowen-Tony/agentkv | agentkv |
| Diffusion language models | Register Tokens for Bounded-State Reasoning in Diffusion Language Models | University of Wisconsin–Madison, 2026-09-14 | No public author repository found as of 2026-09-16. | register-tokens-dllm |
| Attention and long context | SAS: Simple Attention Sparsification via End-to-End Optimization of Context Ranking | Tencent HY LLM Frontier / HKUST (Guangzhou) / HKUST, 2026-09-11 | Yes: official repository | sas-attention |
| Long-video understanding | Caption-once, Frames-on-Demand: Visual-Need Routing for Budget-Aware Agentic Long Video Understanding | Queen Mary University of London, 2026-09-10 | No public author repository was found on 2026-09-14. | frames-on-demand |
| MoE | Data Scarcity and Model Sparsity: Mixtures-of-Experts Overfit More to Repeated Data | Stanford University, 2026-09-10 | No public author repository found as of 2026-09-14. | repeat-aware-moe |
| Optimizers | Musec: MomentUm SpEctral Clipping for Stable Muon-type Training | National University of Singapore, 2026-09-10 | No public author repository was found on 2026-09-14. | musec |
| Multimodal reasoning | OmniKVQuant: KV Cache Quantization for Omni-LLMs | KAIST, 2026-09-10 | OmniKVQuant. | omnikvquant |
| Unified multimodality | SenseNova-U1.5: Towards Native Unified Visual Intelligence | SenseTime Research, 2026-09-10 | SenseNova-U1. | sensenova-u1-5 |
| Model routing | SWRouter: Similarity-Contractive Window Routing for Multi-Turn Large Language Model Conversations | Shanghai Jiao Tong University, 2026-09-10 | No public author repository found on 2026-09-14. | swrouter |
| Inference and systems efficiency | BeaconKV: Key-Value Cache Compression Guided by Beacon Queries for Efficient Large Reasoning Model Inference | Hanyang University, 2026-09-04 | No public author repository identified (checked 2026-09-07). | beaconkv |
| Inference and systems efficiency | KVMem: Virtualizing Million-Token Agent Workspaces on a Consumer GPU | Shanghai University of Finance and Economics, 2026-09-04 | Yes: official author code. | kvmem |
| Model architecture | Lngram v2: Latent N-Gram Memory with Interpretable Discrete Representations | Beijing University of Posts and Telecommunications / Kuaishou Technology, 2026-09-03 | No public author code was found on 2026-09-06. | lngram-v2 |
| Inference and systems efficiency | Random Attention: Rethinking KV Cache Eviction for Efficient Reasoning | Salesforce AI Research / University of Illinois Urbana-Champaign, 2026-09-03 | Public implementation | random-attention |
| In-context learning from weak-model failures | CritICL: Inference-Time Weak-to-Strong Generalization from Small Language Model Failure Modes | The Ohio State University, 2026-08-27 | The paper links CRITICL, but the repository was empty on 2026-08-31 and was not counted as a release. | criticl |
| Multimodal foundation models | PACE: A Unified Condense-and-Extract Paradigm for Fast VLM Inference | Sun Yat-sen University, 2026-08-27 | Official author code | pace-vlm |
| Attention and long context | TwinKV: A Composable Repair Pass for KV Cache Eviction via Pairwise Key Redundancy | The Hong Kong University of Science and Technology (Guangzhou), 2026-08-27 | No author code was released according to the 2026-08-29 check. | twinkv |
| Multimodal foundation models | MLLMCLIP: Feature-Level Distillation of MLLM for Robust Vision-Language Representations | KAIST / Sony AI, 2026-08-26 | No public author code was found on 2026-08-28. | mllmclip |
| Multimodal foundation models | VBVR-Pro: A Scalable and Verifiable Suite for Native Visual Reasoning | Nanyang Technological University / VBVR Community, 2026-08-26 | Author project and code. | vbvr-pro |
| Multimodal foundation models | WeMM-Embedding: WeChat Multi-Modal Embedding Technical Report | WeChat Vision, Tencent, 2026-08-25 | Official author code | wemm-embedding |
| Model architecture | RARE: Decoupling Representation Steering from Expert Routing in Mixture-of-Experts Language Models | Huazhong University of Science and Technology, 2026-08-21 | No official or author-released code found in the paper (checked 2026-08-24). | rare |
| Pretraining and data | Fast A/B/n Testing: Exact Multi-Policy Comparison via Tree-Coupled Feedback Sharing | New York University, 2026-08-13 | No public author code found as of 2026-08-24. | tcab |
| Multimodal foundation models | Generation as Auxiliary Supervision: Enhancing Visual Understanding at Zero Inference Overhead via Decoupled Embedding Prediction | ByteDance, 2026-08-12 | No official author code identified (checked 2026-08-13). | gas |
| Attention and long context | Cracks in the Foundation: Seemingly Minor Architectural Choices Impact Long Context Extension | Ai2, 2026-08-10 | allenai/olmpool. | olmpool-long-context |
| Inference and systems efficiency | DistillCache: KL-Guided Adaptive KV-Cache Eviction for Memory-Efficient LLM Inference | Oklahoma State University, 2026-08-09 | No public author code was found as of 2026-08-24. | distillcache |
| Attention and long context | Autonomy-of-Heads: Data-Free Sparse Attention from Frozen Query-Key Geometry | Institute of Computing Technology, Chinese Academy of Sciences, 2026-08-07 | No public author code found as of 2026-08-24. | autonomy-heads |
| Inference and systems efficiency | BaKron: Efficient Quantization with Kronecker-Factored Hessians | University of California, San Diego, 2026-08-06 | No official author code linked by the paper as checked on 2026-08-09. | bakron |
| Evaluation and reliability | Detecting Safety Training Modification in Language Models via Activation Analysis | Google Cloud, 2026-08-06 | GoogleCloudPlatform/activation-model-scanner, Apache-2.0, pinned to 7ae56d3c94d49ca023eb23ef79a241b72b89b35d |
activation-model-scanner — Low-cost component only; full experiments not reproduced. |
| Pretraining and data | Hierarchical Latent Prediction for Language Models | University of Texas at Austin, 2026-08-06 | The paper did not provide official or author code when checked on 2026-08-09. | hilp |
| Model architecture | MACRO: Markov Chain Routing of Transformer Layers | Heinrich Heine University Düsseldorf, 2026-08-06 | Public implementation | macro |
| Inference and systems efficiency | DBLast: Dependent Block Drafting for Stochastic Speculative Decoding | Huawei Technologies Canada, 2026-08-05 | No official author code in the paper when checked on 2026-08-09. | dblast |
| Inference and systems efficiency | QEvict: Recoverable Quantized KV Eviction for Attention-Drift-Robust Long-Context Decoding | Indian Institute of Technology Roorkee, 2026-08-05 | No official author code in the paper when checked on 2026-08-09. | qevict |
| Multimodal foundation models | Towards Physics of Multimodal Pretraining: Knowledge Flow, Modality Synergy, Early Unification, and Recipes | Meta, 2026-08-05 | No public author code found as of 2026-08-24. | physics-mm-pretraining |
| Inference and systems efficiency | TaskPress: Query-Agnostic KV Cache Compression via Task-Guided Pruning | Google, 2026-08-04 | The paper builds on kvpress but does not provide TaskPress-specific source or a verifiable license for it. Third-party kvpress code/license is not author method code. Paper: arXiv non-exclusive license. | taskpress — Low-cost component only; full experiments not reproduced. |
| Model architecture | DART: Decoded Attention over Recurrent States for Efficient Long-Context Sequence Modeling | Zhejiang University, 2026-08-03 | No public author code was found as of 2026-08-24. | dart |
| Attention and long context | Learning What to Remember: Test-Time Training via Context Distillation | Princeton University, 2026-08-03 | dangxingyu/ttcd. | ttcd |
| Model architecture | Role-Decoupled Attention Residuals | Kehan Wang (no affiliation listed), 2026-08-03 | No official author code in the paper when checked on 2026-08-09. | rd-attnres |
| Model architecture | TransMem: Transforming Hidden States into Memory for Large Language Models | Authors did not disclose affiliation, 2026-07-31 | No public author code found as of 2026-08-24. | transmem |
| Multimodal foundation models | ReToken: One Token to Improve Vision–Language Models for Visual Retrieval | UIUC / Microsoft Research / Google DeepMind, 2026-07-30 | Official author code | retoken |
| Inference and systems efficiency | WIDE: Boosting Adaptive LLM Inference via Token-level Dynamic Width Pruning | EIT-NLP / LMU Munich, 2026-07-30 | Public implementation | wide |
| Model architecture | Penelope: Localized Latent Recurrence for Efficient Structured Reasoning | Academic author team, 2026-07-28 | The paper did not supply official or author code when checked on 2026-08-09. | penelope |
| Pretraining and data | DataOrchestra: Learning to Orchestrate Per-Example Curation of Pretraining Data | Fudan University / Shanghai Jiao Tong University / SII-GAIR, 2026-07-27 | Public author implementation. | data-orchestra |
| Inference and systems efficiency | Adaptive Depth Sparse Framework: Similarity-Driven Resource Allocation for Pre-Trained LLMs | Huawei ACS Lab / Southern University of Science and Technology, 2026-07-23 | No official or author implementation is linked by the paper (checked 2026-08-09). | adadsf |
| Attention and long context | Anti-Periodic Positional Encoding: Möbius Boundary Conditions Make In-Context Retrieval Reliable | Independent researcher, 2026-07-23 | No official author code in the paper when checked on 2026-08-09. | mobius-rope |
| Inference and systems efficiency | Hilbert Operator for Progressive Encoding (HOPE): A Mathematical Framework for Deconstructing Learned Representations in Deep Networks | Google DeepMind, 2026-07-23 | google-deepmind/hope, pinned to f3c4b251d0c9dae05fb165b9bc645a5a52955e62 |
hope-hilbert — Low-cost component only; full experiments not reproduced. |
| Model architecture | Naju: A Native Discrete State-Space Model with Independent Retention and Writing for Long-Sequence Memory | Independent researchers, 2026-07-23 | No official or author-released code found in the paper (checked 2026-08-09). | naju |
| Attention and long context | Parameter-free Adaptive Sparse Attention via Compression-Based Content Selection | Pennsylvania State University, 2026-07-23 | No official or author-released code found in the paper (checked 2026-08-09). | gzip-sparse-attention |
| Inference and systems efficiency | Windowed-MTP: Removing the Full-Context Draft-KV Tax at Million-Token Context | NVIDIA, 2026-07-23 | Official author code | windowed-mtp |
| Inference and systems efficiency | GaugeQuant: Online Learning of Quantization-Optimal Bases from LLM Symmetries | University of Cambridge, 2026-07-22 | MPedraBento/gauge-quant. | gaugequant |
| Model architecture | Convolution for Large Language Models | Huawei / Peking University / Tsinghua University, 2026-07-20 | No official author code in the paper when checked on 2026-08-09. | conv-llm |
| Inference and systems efficiency | C²KV: Compressed and Composable KV Cache Reuse for Efficient LLM Inference | Shanghai Jiao Tong University, 2026-07-20 | s7a9/C2KV. | c2kv |
| Pretraining and data | PPL-Factory: Task-Aware and Budget-Aware Data Selection from Language Modeling to Reasoning | McGill University, 2026-07-20 | No official or author-released code found in the paper (checked 2026-08-09). | ppl-factory |
| Pretraining and data | OpenLanguageModel: Readable and Composable Small-Language-Model Pretraining for Education and Research | Indian Institute of Technology Madras, 2026-07-18 | Official author code | open-language-model |
| Attention and long context | Looped Latent Attention: Cross-Loop KV Compression for Looped Transformers | University of Maryland / Meta AI, 2026-07-16 | The paper did not provide official or author code when checked on 2026-08-09. | looped-latent-attention |
| Multilingual reasoning and evaluation | Cross-Lingual Exploration for Parametric Knowledge | The Hebrew University of Jerusalem, 2026-06-23 | No author code link found in the pinned v1 PDF or abstract. The authors' personal pages link the paper and model/paper indexes, not author source. | Planned key cross-lingual-exploration — Not reproduced due to cost (review complete); research material only, no executable entry. |
| Attention and long context | MiniMax Sparse Attention | MiniMax, 2026-06-11 | MSA. | minimax-sparse-attention |
| Continual learning | Language Models Need Sleep: Learning to Self-Modify and Consolidate Memories | Google Research, 2026-06-02 | No author code link found in the pinned full text, abstract, or official blog. Do not confuse this with the HOPE repository for a different Hilbert compression paper. | sleep-lti — Low-cost component only; full experiments not reproduced. |
| Cross-language evaluation | XLGoBench: Detecting cross-lingual skill gaps with algorithmic tasks | Google DeepMind, 2026-05-29 | Author supplement v1 on Zenodo, file MD5 1fd5dd56c216a407dd5a13f08b0f1065; no Git SHA. Only the paper and public metadata were checked. The independently written scorer does not copy templates. The Zenodo page has no clear license and the 82.2 MB package was not downloaded to inspect internal licensing. Paper: arXiv non-exclusive license. |
xlgobench — Low-cost component only; full experiments not reproduced. |
| Model architecture | Memory Grafting: Scaling Language Model Pre-training via Offline Conditional Memory | Tsinghua University / Microsoft Research Asia, 2026-05-20 | No official or author-released code found in the paper (checked 2026-08-09). | memory-grafting |
| Enterprise LLMs | Customizing an LLM for Enterprise Software Engineering | Google, 2026-05-15 | No public author implementation was found as of 2026-10-02. | gemini-for-google |
| Model routing | Optimized Deferral for Imbalanced Settings | Google Research, 2026-04-30 | No author implementation link found in the reviewed paper. | mild — Low-cost component only; full experiments not reproduced. |
| Model evaluation | ProEval: Proactive Failure Discovery and Efficient Performance Estimation for Generative AI Evaluation | Google DeepMind, 2026-04-25 | google-deepmind/proeval, public repository linked by the paper. Upstream training is not run locally. | proeval — Low-cost component only; full experiments not reproduced. |
| Inference and systems efficiency | FUSE: Ensembling Verifiers with Zero Labeled Data | Stanford University, 2026-04-20 | Appendix G releases raw HLE/IMO data and says the pipeline will be open sourced; no full author pipeline was verified. HazyResearch/scaling-verification is baseline code, not FUSE. This is not proof of non-release. | Planned key fuse-verifiers — Not reproduced due to cost (review complete); research material only, no executable entry. |
| Attention and long context | Switch Attention: Towards Dynamic and Fine-grained Hybrid Transformers | Peking University / Huawei Technologies, 2026-03-27 | No official author code linked as of 2026-08-09. | switch-attention |
| Generative retrieval | Efficient, Property-Aligned Fan-Out Retrieval via RL-Compiled Diffusion | University of Illinois Urbana-Champaign / Google Research, 2026-03-06 | No public author repository found as of 2026-09-16. | r4t |
| Pretraining and data | Theoretical Perspectives on Data Quality and Synergistic Effects in Pre- and Post-Training Reasoning Models | University of Southern California / Google Research, 2026-03-01 | No implementation repository found in v1 or on the official institution page; this is not an exhaustive claim about public code. | data-quality-lsa — Low-cost component only; full experiments not reproduced. |
| Model architecture | A Theoretical Framework for Modular Learning of Robust Generative Models | Google Research, 2026-02-19 | No implementation repository found or released in v1 or the official institution page. This is not a claim that no code exists anywhere. | modular-robust-generative — Low-cost component only; full experiments not reproduced. |
| Pretraining and data | ECO: Quantized Training without Full-Precision Master Weights | Google Research / ISTA, 2026-01-29 | No author repository was found in the paper or official Google details. This is not proof of non-release. | Planned key eco-optimizer — Not reproduced due to cost (review complete); research material only, no executable entry. |
| Pretraining and data | Mobility-Embedded POIs: Learning What A Place Is and How It Is Used from Human Movement | University of Southern California, 2026-01-29 | No training repository found in v1 or on the official institution page; this is not a claim that no implementation exists anywhere. | me-pois — Low-cost component only; full experiments not reproduced. |
| Model architecture | Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models | DeepSeek, 2026-01-12 | deepseek-ai/Engram. | engram |
| Model architecture | mHC: Manifold-Constrained Hyper-Connections | DeepSeek-AI, 2025-12-31 | No official or author-released code found in the paper (checked 2026-08-09). | mhc |
| Safety and verifiability | Detecting Post-generation Edits to Watermarked LLM Outputs via Combinatorial Watermarking | University of Minnesota, 2025-10-02 | No method repository found in the pinned paper or exact-title search; code SHA and license could not be verified. Local equations are implemented independently. Paper license: CC BY 4.0. | combinatorial-watermark — Low-cost component only; full experiments not reproduced. |
| Dynamic candidate classification | GLiClass: Generalist Lightweight Model for Sequence Classification Tasks | Knowledgator, 2025-08-11 | Knowledgator/GLiClass | gliclass |
| Interpretability | Model Directions, Not Words: Mechanistic Topic Models Using Sparse Autoencoders | Columbia University, 2025-07-31 | Public author resources, pinned to 2faface4bd07985f885935d62a11d5e4bf9c73bf, verified MIT. The local MIT mETM and steering components independently implement the pinned v3 equations, without copying upstream code. |
mechanistic-etm — Low-cost component only; full experiments not reproduced. |
| Attention and long context | Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free | Qwen / Alibaba, 2025-05-10 | Yes: official author code. | gated-attention |
| Multimodal foundation models | SmolVLM: Redefining small and efficient multimodal models | Hugging Face, 2025-04-07 | huggingface/smollm. | smolvlm |
| Pretraining and data | Muon is Scalable for LLM Training | Moonshot AI / UCLA, 2025-02-24 | Yes: official author code. | muon |
| Multimodal foundation models | SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features | Google DeepMind, 2025-02-20 | Public implementation | siglip2 |
| Attention and long context | MoBA: Mixture of Block Attention for Long-Context LLMs | Moonshot AI, 2025-02-18 | MoonshotAI/MoBA. | moba |
| Attention and long context | Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention | DeepSeek, 2025-02-16 | No official or author-released code found in the paper (checked 2026-08-09). | native-sparse-attention |
| Model architecture | Byte Latent Transformer: Patches Scale Better Than Tokens | Meta FAIR, 2024-12-13 | facebookresearch/blt. | blt |
| Model architecture | Hymba: A Hybrid-head Architecture for Small Language Models | NVIDIA, 2024-11-20 | NVlabs/hymba. | hymba |
| Pretraining and data | Data Mixing Laws: Optimizing Data Mixtures by Predicting Language Modeling Performance | University of Cambridge / Shanghai AI Laboratory, 2024-03-25 | yegcjs/mixinglaws. | data-mixing-laws |
| Inference and systems efficiency | Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads | Together AI / Princeton University / University of Illinois Urbana-Champaign, 2024-01-19 | FasterDecoding/Medusa. | medusa |
| Model architecture | Mamba: Linear-Time Sequence Modeling with Selective State Spaces | Carnegie Mellon University / Princeton University, 2023-12-01 | Public implementation | mamba |
| Inference and systems efficiency | AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration | MIT / NVIDIA / Harvard / SJTU, 2023-06-01 | Authors' code. | awq |
| Attention and long context | GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints | Google Research, 2023-05-22 | No official author code linked as of 2026-08-09. | gqa |
| Pretraining and data | DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretraining | Stanford University / Google Research, 2023-05-17 | sangmichaelxie/doremi. | doremi |
| Multimodal foundation models | Visual Instruction Tuning | University of Wisconsin-Madison / Microsoft Research / Columbia University, 2023-04-17 | LLaVA. | llava |
| Multimodal foundation models | BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models | Salesforce Research, 2023-01-30 | salesforce/LAVIS. | blip2 |
| Inference and systems efficiency | Fast Inference from Transformers via Speculative Decoding | Google Research, 2022-11-30 | Google Research speculative_decoding. | speculative-decoding |
| Attention and long context | Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation | University of Washington / Meta AI, 2021-08-27 | Author code. | alibi |
| Attention and long context | RoFormer: Enhanced Transformer with Rotary Position Embedding | Zhuiyi Technology, 2021-04-20 | Public implementation | rope |
| Multimodal foundation models | Learning Transferable Visual Models From Natural Language Supervision | OpenAI, 2021-02-26 | openai/CLIP. | clip |
| Model architecture | Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity | Google Brain, 2021-01-11 | Author implementation in Mesh TensorFlow. | switch-transformer |
Browse by: