Video-RSI: Recursively improve video agents through harness evolution

简体中文

L1 acceptance rule, not automatic end-to-end video harness evolution.

Paper information

Field	Details
Paper	Video-RSI
Institution	Tsinghua University (first author).
First public release	2026-09-29 (arXiv v1)
Author code	bingjunluo/Video-RSI.
Local adapter or method key	video-rsi
Local reproduction code	src/auto_research/agent_research/latest_20260930.py

Paper method and evidence

Existing traces contain only observations selected by the current harness, so alternative failure explanations can remain indistinguishable. Revisit training videos to collect additional evidence, test diagnoses and revise the harness. Accept only a Pareto improvement in accuracy or visual cost.

flowchart LR
  H[Current harness] --> E[Run video task]
  E --> D[Diagnose failures]
  D --> V[Revisit video for evidence]
  V --> R[Revise harness]
  R --> A[Check accuracy and visual cost]
  A -->|Pareto acceptance| H

Original paper figure

Figure 1 contrasts direct answers, a fixed agent and recursive harness improvement. From the original paper; copyright remains with the authors.

Core equation

Accept when \(a'\ge a-\epsilon_a\) and \(c'\le c+\epsilon_c\) and at least one dimension strictly improves. A weighted scalar must not hide material regression.

Reported results

The paper reports improved video understanding accuracy with fewer processed frames. No local benchmark replication.

Local reproduction

Three-seed acceptance diagnostic. Candidate harnesses do not see private training answers or example-level scores. No real video decode, code-edit sandbox, generation stopping rule, or executable Evolve operator.