Token-Level Video Reinforcement Learning¶
L1 core-mechanism diagnostic. Executes gradient-credit normalization without training a video diffusion model.
Paper information¶
| Field | Details |
|---|---|
| Paper | Token-Level Video Reinforcement Learning |
| Organization | No first-author affiliation is listed on the paper's first page. |
| First public release | 2026-10-01 (arXiv v1) |
| Authors' code | No public author implementation was found as of 2026-10-03. |
| Adapter / method | tvrl |
| Local reproduction code | src/auto_research/post_training/latest_20261003.py |
Original paper¶
Motivation and method¶
A frozen VLM's video-input gradients identify which generated tokens most affect reward. Group advantage is then redistributed densely to tokens.
flowchart LR
I[Public input and current state] --> M[tvrl mechanism]
M --> A[Auditable intermediates]
A --> O[Output or state update]Key figure from the paper¶
Figure 1: Training pipeline and key optimization steps. From the original paper; copyright remains with the authors. Click the image for the source.
Core equation¶
\(c_t=\|\partial R/\partial z_t\|/mean_t\|\partial R/\partial z_t\|\).
Paper results¶
Video-generation evaluations report better prompt following and temporal quality than scalar GRPO.
Local reproduction¶
Local comparison: The baseline disables the mechanism or uses default state; the treatment executes the defining operator. L1 reports no formal relative improvement, and relative percentages are not applicable.
- Three-seed diagnostic
diagnostic_only=true, ineligible for formal capability ranking.
Reproduction boundaries¶
Gradient-credit normalization runs locally; no video diffusion model was trained.