AgentOPSD: Recursive Bayesian credit for pivotal turns

简体中文

Core turn evidence, recursive belief, pivotal credit and critic-free update.

Paper information

Field	Details
Paper	Paper or original source
Institution	Tsinghua University / Zhejiang University / Meituan.
First public release	2026-08-06 (arXiv v1)
Author code	ZethWang/AgentOPSD is public; README says complete code is pending.
Local adapter or method key	agent-opsd
Local reproduction code	src/auto_research/agent_research/

Paper method and evidence

Aggregate privileged-replay teacher/student log-prob gaps into turn evidence. Update success belief recursively in log-odds space and use adjacent revisions to identify pivotal turns.

flowchart LR
  A[Aggregate privileged token gap by turn] --> B[Update success log-odds]
  B[Update success log-odds] --> C[Credit pivotal revisions]

Original paper figure

The original figure shows recursive turn-credit estimation. From the original paper; copyright remains with the authors.

Core equation

\[ e_t=\sum_{j\in turn_t}(\log\pi_T(y_j)-\log\pi_S(y_j)),\quad \ell_t=\ell_{t-1}+e_t,\quad c_t=\sigma(\ell_t)-\sigma(\ell_{t-1}). \]

Reported results

Qwen2.5-7B reaches 89.1% ALFWorld success, ahead of GRPO and self-distillation baselines. No production A/B.

Local reproduction

PlanBench mini, 120 episodes, seed 42: joint success 1.0000, mean cost 0.9000, nonzero mechanism telemetry. Metrics. A public deterministic mini-suite verifies state updates at matched budget, not paper LLMs, multi-GPU RL, private environments or complete benchmarks.

auto-research agent-eval --method agent-opsd --benchmark planbench-mini --episodes 120 --seed 42
auto-research evolve --model agent --dataset planbench-mini --direction "Combine agent-opsd with installed paper operators" --generations 2 --population 4