Multimodal model method index

简体中文 · English

This index includes implemented papers and high-cost research material. Listing does not imply implementation. See paper write-ups and receipts for public-data runs and GPU validation status. The mapping from backbone connectors to paper-derived genomes is listed here as well.

Implemented papers and research materials

Method family	Paper / official model	Institution and first publication	Adapter / standalone component
Image matching and sparse correspondence	FlexGlue: Sparse-to-Dense Feature Matching and Visual Disambiguation	ETH Zurich, 2026-10-23	Not reproduced (high-cost research material)
Unified action representations and conditional attention	UniAction: Unified Action Conditioning for World Models	Stanford University, 2026-10-23	uniaction
Joint code, video, and action generation	Agentic World Action Models	UC Berkeley, 2026-10-21	Not reproduced (high-cost research material)
Single-pass structured reranking	Multimodal Reranking Is a System One Decision	Aalto University, 2026-10-20	system-one-rerank
Visual evidence-constrained distillation	CV-OPD: Contrastive Vision On-policy Distillation	Xiamen University, 2026-10-09	cv-opd
Visual token compression and dynamic evidence recovery	Look Back, Think Ahead: Visual Memory on Demand for Efficient Multimodal Reasoning	City University of Hong Kong, 2026-10-08	vimod
Frozen intermediate probes and verifier bootstrapping	Unsupervised Multimodal Reward Modeling	Massachusetts Institute of Technology, 2026-10-08	umrm
Visual token compression and dynamic evidence recovery	V-CoLA: Vision Token Compression with Linear Attention	Alibaba Cloud Computing, 2026-10-08	vcola
Visual state-transition supervision	WOVEN: Weaving Visual World Modeling into Multimodal LLMs	Northwestern University, 2026-10-08	woven
Visual tokens and cross-modal retrieval	EmbeddingGemma 2: Unified and Efficient Multimodal Representations	Google, 2026-10-06	embedding-gemma2
Visual tokens and cross-modal retrieval	StackTok: Accelerating VLMs Inference with Budget-Adaptive Visual Token Selection	Sichuan University, 2026-09-15	stacktok
Video understanding and token compression	VideoMM: Adaptive Macro-Micro Inference for Efficient Video MLLMs	University of Science and Technology of China, 2026-09-15	videomm
Audio description and contrastive ranking	From Visual Cues to Spoken Narration: Rethinking Audio Description	TU Darmstadt & hessian.AI, 2026-09-01	cue2narrate-drl
Generative auxiliary supervision for better understanding	Generation as Auxiliary Supervision: Enhancing Visual Understanding at Zero Inference Overhead via Decoupled Embedding Prediction	ByteDance, 2026-08-12	gas
Multi-instance editing and isolated attention	Editing Everything Everywhere All at Once	University of Modena and Reggio Emilia, 2026-06-30	mice
Unified video embedding evaluation	MVEB: Massive Video Embedding Benchmark	Harvard University, 2026-06-12	Not reproduced (high-cost research material)
Low-resource speech translation evaluation	NaijaS2ST: A Multi-Accent Benchmark for Speech-to-Speech Translation in Low-Resource Nigerian Languages	Mila - Quebec AI Institute, 2026-04-17	Not reproduced (high-cost research material)
Efficient visual token compression	SmolVLM: Redefining small and efficient multimodal models	Hugging Face, 2025-04-07	smolvlm
Contrastive pretraining and self-distillation	SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features	Google DeepMind, 2025-02-20	siglip2
Visual tokens and cross-modal retrieval	Visual Instruction Tuning	University of Wisconsin-Madison, 2023-04-17	llava
Cross-modal connectors and frozen backbones	BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models	Salesforce Research, 2023-01-30	blip2
Visual tokens and cross-modal retrieval	Learning Transferable Visual Models From Natural Language Supervision	OpenAI, 2021-02-26	clip

Paper operators available to Evolve

Genome	Source	Main change
micro_vlm_linear	Basic CLIP / LLaVA projection	Linear connector retaining all patch tokens
micro_vlm_mlp	LLaVA	Two-layer nonlinear projector
micro_vlm_qformer	BLIP-2	Learned query cross-attention, 16 → 4 tokens
micro_vlm_pixelshuffle	SmolVLM	2×2 space-to-depth, 16 → 4 tokens
objective:siglip2	SigLIP 2	Sigmoid image–text objective and masked-view consistency

Browse by: Browse by institution, company, or university · Browse by topic · Browse by year