CAMIE: Align multimodal item embeddings using co-engagement

简体中文

Core mechanism on public data: content vectors, co-engagement positives, and symmetric retrieval scoring.

Paper information

Field	Details
Paper	arXiv 2608.30255
Institution	Snap Inc.
First public release	2026-08-31 (arXiv v1)
Author code	No public author implementation found as of 2026-09-05.
Local adapter or method key	camie
Local reproduction code	src/auto_research/reproductions/camie/

Paper method and evidence

Encode images and metadata with one LLM/MLLM interface. Mine jointly engaged items from user journeys, then use symmetric in-batch InfoNCE to shift content similarity toward behavioral co-conversion. One checkpoint supports both multimodal and text-only retrieval.

flowchart LR
  X[Images and metadata] --> E[Shared embedding interface]
  J[User journeys] --> P[Co-engagement positives]
  P --> L[Symmetric in-batch InfoNCE]
  E --> L

Original paper figure

The original figure shows the item encoding and contrastive training pipeline. From the original paper; copyright remains with the authors.

Core equation

\(\mathcal L=-\frac12[\log\frac{e^{q_a^Tk_b/\tau}}{\sum_j e^{q_a^Tk_j/\tau}}+\log\frac{e^{q_b^Tk_a/\tau}}{\sum_j e^{q_b^Tk_j/\tau}}]\).

Reported results

Author-reported deployed production result: CTR +0.211% and CVR +1.911% on overall traffic.

Local reproduction

Use public MovieLens content and co-occurrence transitions as proxies. Local NDCG@10 is 0.05121 versus baseline 0.05401, a -5.18% change. Three-seed metrics retain the negative result. No private DPA journeys, production MLLM checkpoint, or ANN serving; CPU mechanism only.