Agent paper research¶
Recent scaled mechanism experiment: GRAFT trajectory-graph credit assignment. LLM-policy and formal-environment evaluation are not complete.
The paper implementation and evaluation library covers agent memory, planning, tools, collaboration, and self-evolution here. Deterministic mini-suites check state and reuse across episodes. swebench-local creates temporary repositories, edits code, and executes regression tests. These methods and evaluators also underpin Agent Evolve.
Reproduction fidelity
Mini-suites reproduce mechanisms. Code sandboxes execute real actions in a microbenchmark, not official SWE-bench Lite. Paper pages separate author results, local mapping, and limits.
Quick entry points¶
- AgentX-Model: Dual proposer/investigator roles, four research actions, and outcome-blind dependency-graph replay. Public MovieLens-1M mechanism check, not the private production system.
- AtomRec: Atomic preference memory, semantic–collaborative links, and multi-hop recommendation evidence.
- CoSkill: Joint training of reasoning policy, meta-skill policy, and a hierarchical skill library.
- SiLR: Shadow execution, per-branch product-order admission, and same-source process rewards.
- Multi-Harness RL: Freezes experience to distinguish grouping credit from transfer across execution harnesses.
- DRACO: Dynamic rubrics and closed-form redistribution of step credit.
- Agents in Evolve: Supported status, combination axes, and execution limits.
- Methods index: Browse memory, planning, tool management, and other areas.
- Lineage and gaps: Audits the classic lineage, recent coverage, and real-environment prerequisites.
- Shared evaluation protocol: Mini-suites, cost definitions, fair comparisons, and inclusion criteria.
- L2.1 isolated capability evaluation and Evolve: Removes guide/oracle, isolates train/validation/test, and integrates nine Agent genome axes into the multi-round controller.
- Trainable policies and real executors: Agent Lightning checkpoint updates and a matched-budget execution matrix.
- SPO++: Freezes prompt values at generation and corrects asynchronous advantages under an action-token measure.
- SkillForge: Explicit creation, invocation, verification, and revision of reusable skills.
- AHEAD: Adds environmental feedback to ordinary steps and corrective hints to erroneous steps.
- SMITH: Jointly optimizes tool creation and use, separating schema, code, and result verification.
- JIT-Agent: Builds task-specific memory, planning, action protocol, and tools/skills; repairs the harness from failures.
- TraceML: Converts execution traces into trainable state, action, and credit records for cross-task diagnosis and reuse.
- AdaVDR: Allocates retrieval depth by verification uncertainty and constrains answers with evidence consistency.
- TOPAS: Typed observation, planning, and action schemas for long-horizon tools.
- CaSKG: Causal skill graphs for retrieving, composing, and validating task-specific paths.
- ProgRouter: Progressively routes planner, tools, and recovery policy from the task and live feedback.
- Agent-G²: Estimates Gaussian guidance bands online from existing rollouts, without extra probes.
- AutoSaddler: Diagnoses failed traces and generates and verifies durable harness patches.
- Toolformer: Self-supervised tool-call filtering by token-loss improvement.
- Self-Refine: One model generates, critiques, and revises iteratively.
- ReWOO: Separates Planner, Worker, and Solver to reduce repeated observations.
- AutoGen: Role-based multi-agent messages with explicit termination.
- PEARL: Offline tool exploration and planning-centric GRPO.
- Tree of Thoughts: Thought trees, value estimates, and BFS backtracking.
- LATS: MCTS, environmental feedback, and reflective search.
- ReAct: Alternating Thought, Action, and Observation traces.
- Reflexion: Converts failure feedback into verbal reflection for reuse across trials.
- Voyager: Automatic curriculum, execution checks, and a growing skill library.
- U-Mem: Cost-aware active knowledge acquisition and memory verification.
- LEGOMem: Composable orchestrator and executor procedure memory.
- MemTool: Dynamic tool memory under limited context.
- MetaGPT: Product, architecture, engineering, and testing SOP.
- CRITIC: Iterative revision from real tool feedback.
- Agent Lightning: Decoupled execution and training with credit assignment.
- GiGPO: Separately attributes credit to trajectory groups and shared step groups.
- StepPO: Uses environment steps as MDP units for GAE and ratio clipping.
- SWE-agent: Agent–computer interface for software engineering.
- OpenHands: Editor, terminal, and sandbox event stream.
- MRKL: Routes across neural and discrete symbolic experts.
- HuggingGPT: Plans, matches specialist models, executes dependencies, and aggregates outputs.
- Generative Agents: Memory-stream scoring, reflection, and planning.
- MemGPT: Core, working, and archival virtual context with interrupts.
- WebGPT: Browser trajectories, citation constraints, and reward-model rejection sampling.
- SayCan: Language relevance multiplied by skill affordance.
- PAL: Generates a program and runs it with a deterministic interpreter.
- ART: Retrieves demonstrations and pauses and resumes at tool calls.
- SEED: Extracts hindsight skills from completed traces and assigns dense credit.
- CAST: Uses solver-value differences for turn supervision.
- TurnOPD: Adaptive rollout depth and turn-normalized OPD.
- Search-R1: Interleaved reasoning/search, retrieval token masks, and outcome reward.
- RAGEN: StarPO-S trajectory RL with Echo Trap detection and stabilization.
- LOOP: Leave-one-out PPO without a value model; reuses old rollouts with token-level clipping for long interactions.
- WebAgent-R1: Compresses page context dynamically and trains on parallel complete traces with M-GRPO outcome reward.
- MUA-RL: Rollouts include a simulated user for multi-turn intent clarification, real tools, and terminal task reward.
- HiSkill: Hierarchical graph of skills, AtomicOps, and typed edges.
- UniMem: Self-routing episodic/parametric memory with consolidation.
- CAM-DF: Turns a frozen tool ranking into cost-aware prefix stopping.
- SkillRise: Alternates solving related tasks and maintaining skill documentation.
- TAPO: Alternates policy and action-conditioned transition prediction on the same rollout.
- GRSD: Contrasts successful and failed reflections in a group to obtain privileged turn-level guidance.
- EnvACE: Alternates acting and rehearsal in one policy with role-specific group advantages.
- AgentOPSD: Recursively aggregates privileged token gaps into Bayesian credit for key turns.
- OCSD: Compares full versus observation-ablated replay to remove scaffold confounding.
- VerMem: Manages long-term memory, active context, and episode history with a two-level verifier.
- CoEvo-Mem: Alternates updates to retrieval router and memory store.
- OSReward / OS-Shepherd: Audits two-class recall, balanced accuracy, and leniency for cross-platform computer-use rewards.
- DeepResearcher, ReTool, ToolRL: Deep research trajectory rewards and RL for tool selection and execution.
- SAGE, MemSkill, Memento-Skills: Abstract, revise, and retrieve reusable skills from trajectory memory.
- SEARL, Agent0: Joint policy/tool-graph optimization and zero-data multi-agent curricula.
- Agent-R1, CAMEL, ToolBench, GAIA: Step-level Agent RL, role-playing, tool instruction learning, and general public evaluation.
Browse by institution, by topic, or by year.
2026-08-09 P0/P1 additions¶
- EvoHarness-RL, VaG, GSE: Harness policies, validation before skill writes, and global skill-graph evolution.
2026-08-13 MR7 additions¶
- SinkFlex-RL: Combines attention sinks, sliding windows, and critic-free group-relative RL for long-horizon tool agents.
- CIPO, State2State, HindSearch: Evidence credit, environment-derived tasks, and hindsight from failed trajectories.
- HarnessOpt-Bench, CodeGrep, MemoryCPT: Harness optimization evaluation, code retrieval policies, and cost-aware memory.
All nine methods entered the shared runner and compositional Agent Evolve genome, with fixed-seed PlanBench mini-suite results.
2026-08-24 additions¶
- AgentX: Industrial recommendation self-iteration with bounded proposals, repository-constrained implementation, guarded A/B, retained positive and negative outcomes, and SGPO harness updates.
- AUSO: Combines skill internalization, exploration, and bounded utilization through a progressive skill lifecycle and action-level JSD.
2026 historical scan B10–B11¶
- SAPO: Single-Rollout Autoregressive Policy Optimization for Agentic Reinforcement Learning: One autoregressive backbone outputs policy and value under different causal boundaries, combining PPO, on-policy SARSA, and trajectory GAE.
- SPADE: Self-Play in Adaptive Synthetic Executable Environments: One LLM plays environment designer and reasoner. Regret with and without privileged hints learns executable environments near the capability boundary.
- RTPO: Reverse-Turn Policy Optimization for Stabilizing Agentic RL Training: Organizes multi-turn rollouts into a sparse reverse tree and updates turns backwards so decisions stay on-policy with downstream continuations.
- PlanPO: Group Planning-Aware Policy Optimization for Multi-Turn Agentic LLMs: Compares turn count and per-turn response length within successful trajectories for coarse-to-fine planning advantages.
- TRCA: Transition-wise Rubric Credit Assignment for Long-horizon LLM Agents: Constructs base and breakthrough rewards from Evidence, Execution, and Invalidity rubrics at each transition, without successful anchors.
- LoongReflect: Boosting Long-Horizon Reflection in Search Agents via Global Perspective Distillation: Treats reflection and backtracking as memory-control actions on a reversible trajectory tree, combining fast privileged-teacher feedback and slow outcome GRPO.
- HyMem: Hierarchical Context Management for Long-Horizon Agents via Information Isolation: Separates planning, execution, and isolated reasoning. Structured summaries carry progress across context refresh.
- OpenLoopEvolve: A Verifiable Self-Evolution Framework for Loop Policies in Long-Horizon Complex Tasks: Versions observation, planning, memory, action, verification, and recovery assets, using champion–challenger, release monitoring, and rollback.
- Coupling Planning with Episodic Memory in LLM Agents for Software Issue Resolution: Conditions episodic retrieval on a hierarchical phase planner. Detects stuck states from trace statistics, replans, and validates actual execution verdicts.
- ToolLIFT: Lifting Tool-Specific Trajectories into Function-Level Graphs for Generalizable Tool Planning: Lifts tool-specific trajectories into transferable function workflow graphs and separates workflow planning from tool selection.
- HyperAgent: Planning and Acting over Tool-Schema Hypergraphs for Tool-Use LLM Agents: Represents tools as input-to-output schema hyperedges. Builds a Task DAG and expands support graphs for missing state.
- MANTA: Multi-Agent Network Topology Adaptation for Self-Evolving Multi-Agent Systems: Initializes topology from task priors, monitors collaboration traces, and adjusts roles, edges, order, visibility, and verification under bounds.
Research loop¶
flowchart LR
P["Agent papers and hypotheses"] --> I["Shared method interface"]
B["EvoMem / PlanBench / ScaleMCP mini-suite"] --> E["Episode"]
I --> E
E --> A["Agent policy"]
A --> M["Knowledge, procedure, and tool memory"]
M --> O["Answer, plan, or action"]
O --> F["Success, cost, reuse, and eviction"]
F --> A
F --> R["Traces, metrics, and Chinese reports"]
R --> PCurrent implementations¶
| Direction | Method | Mechanism | Local evaluation | Status |
|---|---|---|---|---|
| Environment rehearsal / Agent RL | EnvACE | Act/rehearse roles, private observations, rolewise GRPO | PlanBench mini | Mechanism |
| Fair baseline | Long-context | Full history without memory compression | EvoMem mini | Implemented |
| Tool learning | Toolformer | Candidate API calls filtered by self-supervised loss | ScaleMCP mini | Mechanism |
| Self-refinement | Self-Refine | Generate, receive feedback, and refine | PlanBench mini | Mechanism |
| Decoupled planning | ReWOO | Planner/Worker/Solver and evidence variables | PlanBench mini | Mechanism |
| Multi-agent | AutoGen | Role messages, handoff, critic, and termination | PlanBench mini | Mechanism |
| Planning RL | PEARL | Tool exploration, plan reward, and group update | PlanBench mini | Mechanism |
| Reasoning search | Tree of Thoughts | Thought expansion, value, and BFS/backtracking | PlanBench mini | Mechanism |
| Agent search | LATS | UCT, rollouts, environmental feedback, and reflection | PlanBench mini | Mechanism |
| Reasoning and action | ReAct | Thought → Action → Observation | ScaleMCP mini | Mechanism |
| Self-improvement | Reflexion | Verbal feedback and episodic reflection | PlanBench mini | Mechanism |
| Lifelong learning | Voyager | Curriculum, skill library, and self-verification | PlanBench mini | Mechanism |
| Active memory | U-Mem | Tiered acquisition, semantic retrieval, and Thompson sampling | EvoMem mini | Mechanism |
| Procedural memory | LEGOMem | Cross-episode reuse of orchestration and execution units | PlanBench mini | Mechanism |
| Tool memory | MemTool | Protect workflow tools; evict by recency and success | ScaleMCP mini | Mechanism |
| Multi-agent software engineering | MetaGPT | Four-role SOP and structured artifacts | SWE-style local code | Actual execution |
| Tool feedback | CRITIC | Failed patch, test feedback, and revision | SWE-style local code | Actual execution |
| Agent RL | Agent Lightning | Transitions, credit updates, and policy reuse | SWE-style local code | Actual execution |
| Agent group credit | GiGPO | Macro trajectory and micro step advantages | PlanBench mini | Mechanism |
| Step-aligned Agent RL | StepPO | Step critic, GAE, and within-step ratio aggregation | PlanBench mini | Mechanism |
| Agent transition RL | TAPO | Policy objective and next-observation supervision | PlanBench mini | Mechanism |
| Group-reflective RL | GRSD | Contrast success/failure reflections, stop-gradient guidance, and turn credit | PlanBench mini | Mechanism |
| CUA reward evaluation | OSReward / OS-Shepherd | Success/failure recall, balanced accuracy, and leniency | OSReward mini | Evaluation protocol |
| Software engineering ACI | SWE-agent | Locate, edit, and run regression tests | SWE-style local code | Actual execution |
| General software agent | OpenHands | Editor and terminal event stream | SWE-style local code | Actual execution |
| Neurosymbolic routing | MRKL | Router, neural and symbolic experts, and aggregation | ScaleMCP mini | Mechanism |
| Specialist model orchestration | HuggingGPT | Planning, model selection, and DAG execution | PlanBench mini | Mechanism |
| Memory and reflection | Generative Agents | Recency, importance, relevance, reflection, and plan | EvoMem mini | Mechanism |
| Virtual context | MemGPT | Tiered memory, paging, and interrupts | EvoMem mini | Mechanism |
| Browser QA | WebGPT | Browser trajectories, citations, and rejection sampling | ScaleMCP mini | Mechanism |
| Embodied planning | SayCan | Language relevance multiplied by affordance | PlanBench mini | Mechanism |
| Program reasoning | PAL | Program generation and symbolic runtime | ScaleMCP mini | Mechanism |
| Automatic tool reasoning | ART | Demo retrieval, pause/tool/resume, and library update | PlanBench mini | Mechanism |
| Agentic RL | SEED | Hindsight skills and dense on-policy credit | PlanBench mini | Mechanism |
| Agentic RL | CAST | Solver-value differences and turn credit | PlanBench mini | Mechanism |
| Agentic OPD | TurnOPD | Depth probes, dynamic rollouts, and turn normalization | ScaleMCP mini | Mechanism |
| Search-agent RL | Search-R1 | Multiturn search/reasoning, retrieval loss mask, and outcome reward | ScaleMCP mini | Mechanism |
| Multi-turn Agent RL | RAGEN | Trajectory filtering, critic baseline, and decoupled clipping | PlanBench mini | Mechanism |
| Long-horizon Agent RL | LOOP | Leave-one-out advantage, off-policy reuse, and per-token clipping | PlanBench mini | Mechanism |
| Web-agent RL | WebAgent-R1 | Context compression, parallel trajectories, and M-GRPO | ScaleMCP mini | Mechanism |
| Multi-turn user Agent RL | MUA-RL | Simulated users, intent refinement, real tool responses, and final reward | ScaleMCP mini | Mechanism |
| Hierarchical skills | HiSkill | Skill and AtomicOp nodes with typed-edge subgraphs | PlanBench mini | Mechanism |
| Continual memory | UniMem | Episodic/parametric routing and consolidation | EvoMem mini | Mechanism |
| Tool-acquisition control | CAM-DF | Payoff gap, regret weighting, and cost-sensitive stopping | ScaleMCP mini | Mechanism |
| Cross-task skills | SkillRise | Solve/curate phases with discounted downstream credit | PlanBench mini | Mechanism |
Local experiment snapshot¶
These fixed mini-suite and code-sandbox results describe their mechanisms and execution protocol, not general agent ability. A value of 1 is not proof that a real agent would solve the benchmark.
| Method and benchmark | Joint success | Mean cost | Additional diagnostic |
|---|---|---|---|
| Long-context · EvoMem mini | 1.0000 | 64.5000 | Context cost grows linearly with history |
| U-Mem · EvoMem mini | 1.0000 | 3.0500 | Escalates to tool research after retrieval misses |
| LEGOMem · PlanBench mini | 1.0000 | 1.1200 | 108 procedure-unit reuses |
| MemTool · ScaleMCP mini | 1.0000 | 1.9812 | 200 controlled tool evictions |
| ReAct · ScaleMCP mini | 1.0000 | 3.0000 | 360 reasoning/action steps |
| Reflexion · PlanBench mini | 0.9000 | 1.1000 | 12 reflections, 108 reuses |
| Voyager · PlanBench mini | 1.0000 | 1.1200 | 12 skills, 108 reuses |
| Tree of Thoughts · PlanBench mini | 1.0000 | 2.5000 | 1200 nodes, 480 backtracks |
| LATS · PlanBench mini | 1.0000 | 4.0000 | 480 rollouts, 360 reflections |
| Toolformer · ScaleMCP mini | 1.0000 | 3.0000 | 540 candidates, 360 accepted calls |
| Self-Refine · PlanBench mini | 1.0000 | 2.0000 | 120 feedback and refinement operations |
| ReWOO · PlanBench mini | 1.0000 | 3.5000 | 120 plans, 360 worker calls |
| AutoGen · PlanBench mini | 1.0000 | 3.0000 | 360 cross-role messages |
| PEARL · PlanBench mini | 1.0000 | 1.1200 | 24 exploration operations, 12 policy updates |
| MetaGPT · SWE local | 1.0000 | 3.5000 | 48 role artifacts |
| CRITIC · SWE local | 1.0000 | 4.0000 | 24 rounds of tool feedback and revision |
| Agent Lightning · SWE local | 1.0000 | 3.6250 | Patch reuse 0.7500 |
| SWE-agent · SWE local | 1.0000 | 2.5000 | 12 actual file edits |
| OpenHands · SWE local | 1.0000 | 2.5000 | Editor/terminal event stream |
| MRKL · ScaleMCP mini | 1.0000 | 1.2500 | 360 router calls; 170 symbolic calls |
| HuggingGPT · PlanBench mini | 1.0000 | 2.3500 | 360 model matches; 240 dependency edges |
| Generative Agents · EvoMem mini | 1.0000 | 1.7900 | 354 memories; 30 reflections |
| MemGPT · EvoMem mini | 1.0000 | 0.9200 | 108 writes; 96 page-ins; 108 interrupts |
| WebGPT · ScaleMCP mini | 1.0000 | 3.0000 | 600 references; 240 candidates |
| SayCan · PlanBench mini | 1.0000 | 3.3750 | 1350 affordance checks |
| PAL · ScaleMCP mini | 1.0000 | 1.4000 | 120 programs/interpreter calls |
| ART · PlanBench mini | 1.0000 | 1.5500 | 108 retrievals; 360 pauses |
| SEED · PlanBench mini | 1.0000 | 0.9800 | 12 hindsight skills; 360 dense credit updates |
| CAST · PlanBench mini | 1.0000 | 1.5000 | 2040 solver queries; 360 turn credits |
| TurnOPD · ScaleMCP mini | 1.0000 | 1.3333 | 40 rollout turns saved |
| Search-R1 · ScaleMCP mini | 1.0000 | 2.7500 | 240 queries; 1800 masked tokens |
| RAGEN · PlanBench mini | 1.0000 | 1.6000 | 480 rollouts; 120 filtered; 19 Echo Trap probes |
| LOOP · PlanBench mini | 1.0000 | 2.2000 | 476 old-trajectory reuses; 480 LOO updates; 120 clips |
| WebAgent-R1 · ScaleMCP mini | 1.0000 | 2.6000 | 360 compressed tokens; 120 parallel groups and M-GRPO updates |
| MUA-RL · ScaleMCP mini | 1.0000 | 2.2500 | 360 user turns; 240 intent corrections; 120 terminal rewards |
| HiSkill · PlanBench mini | 1.0000 | 0.6900 | 48 nodes; 60 edges; 324 AtomicOp reuses |
| UniMem · EvoMem mini | 1.0000 | 0.5200 | 24 episodic; 96 parametric; 12 consolidations |
| CAM-DF · ScaleMCP mini | 1.0000 | 4.7333 | Tool exposure -51.02%; 120 early stops |
| SkillRise · PlanBench mini | 1.0000 | 0.6550 | 119 cross-task skill reuses; 360 downstream credits |
| GiGPO · PlanBench mini | 1.0000 | 1.6500 | 36 within-group and 36 between-group advantages |
| StepPO · PlanBench mini | 1.0000 | 0.9000 | 36 step GAE and sequence-ratio aggregations |
| TAPO · PlanBench mini | 1.0000 | 1.0500 | 360 transition targets, accuracy 1.0000 |
| GRSD · PlanBench mini | 1.0000 | 1.3500 | 120 reflection groups, 360 guidance updates |
For metric definitions, see the shared evaluation protocol. Stable artifacts, in order: base mini-suite, classic mini-suite, code sandbox, TAPO/GRSD, classic gaps, P1 candidates, July 29 Agentic RL and memory, classic Agentic RL/OPD, omitted methods, and step-credit algorithms:
- agent-mini-suites-seed42.json
- classic-agent-mini-suites-seed42.json
- agent-code-sandbox-seed42.json
- tapo-grsd-planbench-seed42.json
- p0-missing-agent-mini-suites-seed42.json
- p1-agent-candidates-mini-suites-seed42.json
- agent-20260729-seed42.json
- classic-agentic-rl-opd-seed42.json
- omitted-agentic-rl-opd-seed42.json
- rl-papers-summary-seed42.json
2026-08-29 additions¶
- SWE-Prime: Trajectory-level and semantic-span filtering.
- HarnessLens: Behavior-relevant task selection and attributable verification.
- CoVeMem: Trainable collaborative vector memory.
- SPT: Reference Insert and mid-training of multi-file skill packages.
Batch index: latest-20260829-seed42.json.
Run¶
auto-research agent-eval --method u-mem --benchmark evomem-mini
auto-research agent-eval --method legomem --benchmark planbench-mini
auto-research agent-eval --method memtool --benchmark scalemcp-mini \
--episodes 200 --memory-size 8
auto-research agent-eval --method reflexion --benchmark planbench-mini \
--episodes 120 --memory-size 24
auto-research agent-eval --method pearl --benchmark planbench-mini \
--episodes 120 --memory-size 24
auto-research agent-eval --method critic --benchmark swebench-local \
--episodes 12 --seed 42
Outputs in runs/agent-research/<method>-<benchmark>-seed<seed>/ include step traces, metrics.json, and the Chinese report.md.
Extension requirements¶
New work includes a method implementation, independent write-up, method index, shared-protocol experiments, and stable metrics. Write-ups include metadata, background and changes, architecture, core equations or algorithm, author results, local mapping, experiment, and fidelity limits. The overview maintains global status and comparable snapshots.