简体中文 · English

TCR: Thinking Checklist Residual Reward¶

Implements per-example checklists, an outcome EMA and thinking surplus. The controlled process verifier is not the paper's actual reasoning judge.

Paper information¶

Field	Details
Paper	TCR (arXiv 2607.19824)
Institution	Not listed in the paper
First public release	2026-07-22
Author code	No public repository found as of 2026-07-27.
Local adapter or method key	`tcr`
Local reproduction code	`src/auto_research/post_training/`

Original paper summary¶

Background and main changes¶

Final-answer rewards miss reasoning quality, but adding process reward directly can double-count outcome. TCR constructs a checklist per example and subtracts an exponential moving outcome baseline from process scores. This concentrates updates on reasoning gains unexplained by outcome reward.

flowchart LR
    Y["Rollout"] --> O["Outcome reward"]
    Y --> C["Sample-level checklist"]
    C --> P["Process reward"]
    O --> E["Outcome EMA"]
    P --> S["Thinking surplus"]
    E --> S
    O --> U["Policy update"]
    S --> U

Key figure from the paper¶

Figure 3: Overall workflow, stages and data flow. From the paper; copyright belongs to the authors. Click for the source.

Core equations¶

\[ r_{\mathrm{TCR}}=r_{\mathrm{outcome}} +\alpha\left(r_{\mathrm{process}} -\operatorname{EMA}(r_{\mathrm{outcome}})\right). \]

The residual reduces overlap between outcome and process signals and adapts process rewards to training state.

Offline and online results¶

The paper reports consistent improvements on five models across three families but gives no single headline value in the abstract. It remains under review, with no production A/B reported.

Local reproduction¶

The local checklist verifier produces controlled noisy process scores for candidate reasoning. Training stores outcome EMA and thinking surplus and matches other methods' data, steps and seed.

auto-research post-train --algorithm tcr \
  --dataset gsm8k-candidate --maximum-examples 512 \
  --steps 300 --seed 42

Metric	Untrained policy	TCR
GSM8K candidate accuracy	0.1641	0.8359
Mean reward	0.3126	0.8560
KL(reference)	0.0000	0.5629

See stable metrics.

Scope¶

No free-form generation, model-generated checklists or paper-scale training. This verifies residual process reward on a candidate policy, not the paper's five-model numbers.