TACO: Tail-Token Credit Calibration¶
Core mechanism evidence. Paper and local evaluation are separate.
Paper information¶
| Field | Value |
|---|---|
| Paper | TACO (arXiv 2607.07976) |
| Affiliations | Johns Hopkins University / Rice University / Workato / University of North Carolina at Charlotte |
| First public release | 2026-07-08 |
| Authors' code | Author code. |
| Adapter / method | taco |
| Local code | src/auto_research/post_training/ |
Original paper¶
Background and contribution¶
A correct trajectory's uniform positive advantage can reward implausible low-probability tokens. Estimate contextual tail risk, smoothly reduce high-risk positive credit, and retain negative credit.
flowchart LR
I[Rollout and advantage] --> R[Contextual tail risk] --> G[Positive-credit calibration] --> U[Policy update]Key figure from the paper¶
Original Figure 1: TACO. From the paper; copyright belongs to the authors. Not a local result.
Core mechanism¶
\[
\tilde A_t=\begin{cases}w(\operatorname{tailrisk}_t)A,&A>0\\A,&A\le0,\end{cases}\qquad 0<w(\cdot)\le1.
\]
Paper results¶
The paper reports improvements over GRPO variants on three LLMs and eight benchmarks and more stable long-horizon RL. No online A/B.
Local reproduction¶
Use rollout surprisal as a tail-risk proxy. Positive tail advantage is downweighted continuously; negative advantage is not masked. Metrics.
auto-research post-train --algorithm taco --dataset gsm8k-candidate --maximum-examples 256 --steps 120 --seed 42
Reproduction limits¶
Candidate actions, not token CoT. This does not reproduce the paper's contextual predictor or full-scale RL.