SMES: scalable sparse experts for multitask learning¶

简体中文

Fidelity: core mechanism. Runs the defining mechanism verifiable on public data. Private data, production models, and serving remain outside scope.

Paper information¶

Field	Details
Paper	arXiv 2602.09386
Organization	Kuaishou Technology
First public release	2026-02-10 (arXiv v1)
Authors' code	No public author code was found on 2026-09-05.
Adapter / method	`smes`
Local reproduction code	`src/auto_research/reproductions/smes/`

Original paper¶

Motivation and method¶

Split each task's active experts into a stable subset shared across tasks and an adaptive private subset. Cross-task global load balancing constrains routing, activation count, and expert skew.

flowchart LR
  A["Public behavior and candidates"] --> B["SMES core mechanism"]
  B --> C["Matched-budget method output"]
  A --> D["Common direct baseline"]
  C --> E["Full-catalog ranking and structural diagnostics"]
  D --> E

Key figure from the paper¶

Figure 2: Core architecture, modules, and connections. From the original paper; copyright remains with the authors.

Core equations¶

\[ h_t=\sum_{e\in E_s\cup E_t}g_{t,e}(x)f_e(x),\qquad \mathcal L=\sum_t\mathcal L_t+\lambda\mathcal L_{balance}. \]

Paper results¶

Kuaishou reports online watch time +0.31%, GAUC +0.29%, and around 50% fewer active experts. This is paper evidence, not a local public-data outcome.

Local reproduction¶

Local comparison: Under the same full-catalog protocol, baseline NDCG@10 is 0.05401 and SMES NDCG@10 is 0.05359, -0.78% relative. The local proxy does not extrapolate to online lift.

See the three-seed artifact for complete results, means, standard deviations, and 95% confidence intervals.

auto-research reproduce --paper smes --dataset-dir data --seeds 42,43,44

Reproduction boundaries¶

The local run uses a fixed public MovieLens 100K subset and auditable proxies to validate the core computation. It does not reproduce private logs, production models, online traffic assignment, or serving. No claim is made to reproduce absolute paper metrics or online gains.