CARM: Cancellation-Aware Response Masking for LLM Reinforcement Learning

简体中文

L1 detached response gate; no Qwen3.5 training.

Paper information

Field	Details
Paper	arXiv v1
Institution	Moore Threads AI (first author).
First public release	2026-10-01 (arXiv v1)
Author code	No public author implementation found as of 2026-10-03.
Local adapter or method key	carm
Local reproduction code	src/auto_research/post_training/latest_20261003.py

Paper method and evidence

Take absolute values of per-token current/rollout log-ratios before averaging. Positive and negative drifts can otherwise cancel in a sequence-level geometric mean.

flowchart LR
  R[Current and rollout token probabilities] --> A[Absolute token log-ratios]
  A --> M[Average and exponentiate]
  M --> G[Detached response gate]

Original paper figure

Figure 3 illustrates the cancellation-aware scoring refinement. From the original paper; copyright remains with the authors.

Core equation

\(s=\exp(T^{-1}\sum_t|\log r_t|)\); retain the response only if \(s\le\tau\).

Reported results

The paper reports up to +3.13 points on AIME/BeyondAIME mean@16 and +2.88 pass@1 points across four coding benchmarks.

Local reproduction

Compare against the mechanism disabled or in its default state. The three-seed diagnostic is diagnostic_only=true; no formal relative gain or capability ranking.