AKT-Rec: Asymmetric long-tail knowledge transfer with LLM semantic IDs¶
Fidelity: core mechanism reproduction. Trains LoRA on SmolLM2-135M, with item co-engagement contrastive tuning, future-interest/category supervision, residual semantic IDs, dual views, asymmetric stop-gradient, orthogonality and both activity gates.
Paper information¶
| Field | Details |
|---|---|
| Paper | Paper or original source |
| Institution | Alibaba Group / Peking University |
| First public release | 2026-05-22 (arXiv v1) |
| Author code | No official author code identified (checked 2026-08-09). |
| Local adapter or method key | akt-rec |
| Local reproduction code | src/auto_research/reproductions/akt_rec/ |
Paper method and evidence¶
Sparse, noisy tail representations can degrade frequent entities if they share knowledge symmetrically. AKT-Rec aligns content with collaborative signals through an MLLM and quantizes users and items into hierarchical semantic IDs via RQ-VAE; shared high-level codes define clusters. Ranker features combine cluster and individual views. The stronger InfoNCE direction updates tail vectors against stop-gradient head teachers. One activity gate balances cluster vs individual embeddings; the other fuses instance-level target attention with cluster-level history for CTR.
flowchart LR
A[Co-engagement and user supervision] --> B[Residual semantic IDs]
B[Residual semantic IDs] --> C[Dual representation]
C[Dual representation] --> D[Asymmetric transfer]
D[Asymmetric transfer] --> E[Activity-gated CTR ranking]Original paper figure¶
Figure 1 shows the architecture and relationships between its modules. From the original paper; copyright remains with the authors.
Core equation¶
Reported results¶
On private Tmall data, baseline AUC/GAUC is 0.7510/0.6385 and AKT-Rec gives 0.7536/0.6483; the authors report relative gains of 0.346% and 1.53%, especially on the tail. In a two-week A/B with 10% per arm, clicks rise 2.73%, CTR 2.76%, CTCVR 1.70% and GMV 3.47%.
Local reproduction¶
MovieLens-100K title and genre replace images, descriptions and industrial stats. SmolLM2-135M trains on adjacent positive items with InfoNCE, then learns to predict future-interest vectors and categories from user history; no static embedding substitute. Each positive has one fixed popularity-sampled negative. Tail means fewer than 10 positive training interactions. Both arms share negatives, budget and initialization. The baseline only uses individual embeddings and instance target attention.
| Test variant | AUC | GAUC | Tail AUC |
|---|---|---|---|
| online_base | 0.45600 ± 0.02721 | 0.45208 ± 0.01881 | 0.50854 ± 0.03504 |
| AKT-Rec | 0.47170 ± 0.02199 | 0.47708 ± 0.01031 | 0.51949 ± 0.02081 |
Across three seeds relative gains are 3.44%, 5.53% and 2.15%. Validation AUC goes 0.47915 → 0.48472 and GAUC 0.46458 → 0.47083. Sampled negatives are often more popular than held-out positives, accounting for low absolute scores; compare only within protocol. Training updates 230,400 LoRA parameters, and user loss falls from 3.3054 to 2.1680. Forty autoencoder pretraining steps and staged k-means initialization precede 100 straight-through quantization steps; 1,682 items map to 470 three-level codes, and 320 users to 75 two-level codes, avoiding earlier codebook collapse. Stable metrics. Checkpoints and raw runs are not committed. GME-Qwen2-VL-7B, Qwen3-30B-A3B, industrial RQ-VAE scale/data, Tmall's cluster feature store and online MMoE serving are not reproduced; no production latency or lift claim.