跳转至

Read this page in English

多模态大模型方法索引

本页汇总论文实现与高成本研究资料;收录不等于已实现。公开数据实跑与 GPU 验收状态请查各篇详情和回执。 底座 connector 与论文 genome 的对应关系也在同一处维护。

论文实现与研究资料

方法族 论文 / 官方模型 机构与日期 Adapter / 独立组件
图像匹配与稀疏对应 FlexGlue: Sparse-to-Dense Feature Matching and Visual Disambiguation ETH Zurich,2026-10-23 未复现(高成本资料)
统一动作表示与条件注意力 UniAction: Unified Action Conditioning for World Models Stanford University,2026-10-23 uniaction
代码、视频与动作联合生成 Agentic World Action Models UC Berkeley,2026-10-21 未复现(高成本资料)
单次前向结构化重排 Multimodal Reranking Is a System One Decision Aalto University,2026-10-20 system-one-rerank
视觉证据约束蒸馏 CV-OPD: Contrastive Vision On-policy Distillation Xiamen University,2026-10-09 cv-opd
视觉 token 压缩与动态证据恢复 Look Back, Think Ahead: Visual Memory on Demand for Efficient Multimodal Reasoning City University of Hong Kong,2026-10-08 vimod
冻结中间层探针与验证器自举 Unsupervised Multimodal Reward Modeling Massachusetts Institute of Technology,2026-10-08 umrm
视觉 token 压缩与动态证据恢复 V-CoLA: Vision Token Compression with Linear Attention Alibaba Cloud Computing,2026-10-08 vcola
视觉状态转移监督 WOVEN: Weaving Visual World Modeling into Multimodal LLMs Northwestern University,2026-10-08 woven
视觉 token 与跨模态检索 EmbeddingGemma 2: Unified and Efficient Multimodal Representations Google,2026-10-06 embedding-gemma2
视觉 token 与跨模态检索 StackTok: Accelerating VLMs Inference with Budget-Adaptive Visual Token Selection Sichuan University,2026-09-15 stacktok
视频理解与 token 压缩 VideoMM: Adaptive Macro-Micro Inference for Efficient Video MLLMs University of Science and Technology of China,2026-09-15 videomm
音频描述与对比排序 From Visual Cues to Spoken Narration: Rethinking Audio Description TU Darmstadt & hessian.AI,2026-09-01 cue2narrate-drl
生成辅助监督与理解增强 Generation as Auxiliary Supervision: Enhancing Visual Understanding at Zero Inference Overhead via Decoupled Embedding Prediction ByteDance,2026-08-12 gas
多实例编辑与注意力隔离 Editing Everything Everywhere All at Once University of Modena and Reggio Emilia,2026-06-30 mice
视频嵌入统一评测 MVEB: Massive Video Embedding Benchmark Harvard University,2026-06-12 未复现(高成本资料)
低资源语音翻译评测 NaijaS2ST: A Multi-Accent Benchmark for Speech-to-Speech Translation in Low-Resource Nigerian Languages Mila - Quebec AI Institute,2026-04-17 未复现(高成本资料)
高效视觉 token 压缩 SmolVLM: Redefining small and efficient multimodal models Hugging Face,2025-04-07 smolvlm
对比预训练与自蒸馏 SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features Google DeepMind,2025-02-20 siglip2
视觉 token 与跨模态检索 Visual Instruction Tuning University of Wisconsin-Madison,2023-04-17 llava
跨模态连接器与冻结骨干 BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Salesforce Research,2023-01-30 blip2
视觉 token 与跨模态检索 Learning Transferable Visual Models From Natural Language Supervision OpenAI,2021-02-26 clip

可进入 evolve 的论文算子

Genome 来源 主要变化
micro_vlm_linear CLIP / LLaVA 基础投影 保留全部 patch token 的线性连接器
micro_vlm_mlp LLaVA 两层非线性 projector
micro_vlm_qformer BLIP-2 可学习 query cross-attention,16→4 token
micro_vlm_pixelshuffle SmolVLM 2×2 space-to-depth,16→4 token
objective:siglip2 SigLIP 2 sigmoid 图文目标与 masked-view consistency

分类浏览:按机构/公司/学校 · 按主题 · 按年份